acceptodds
Under review as a conference paper at ICLR 2027

Temporal Action Structure Controls Representation Identifiability in Predictive World Models

Abstract

World models learn from trajectories generated by a behavior policy, yet that policy is often summarized only by marginal action frequencies. We show that the missing temporal information—how actions are composed—can decide whether a predictive objective recovers linear state coordinates or prefers an invertible nonlinear reparameterization of the same state. In a stationary Gaussian controlled world, this choice is governed by a spectral gap between the least predictable linear direction and the most predictable quadratic feature. We prove a sharp phase transition over the full class of standardized square-integrable encoders: a positive gap uniquely selects the state up to rotation and controls near-optimal solutions, whereas a negative gap admits an arbitrarily close, smooth, well-conditioned nonlinear improvement. We then construct two full-support distributions over length-five action sequences that match every interaction among up to three positions and the complete linear prediction operator, but have opposite gap signs. The separation also survives nonlinear Gaussian-stationary dynamics, while an independent construction shows that exact marginal Gaussianization is not sufficient in general. In a controlled visual world, finite-sample spectral learning and a 689K-parameter CNN recover the predicted coordinate switch directly from images. In that same controlled visual world, released LeWorldModel predictor modules retain and use these coordinates; after fresh heads receive equal coverage of all 32 action sequences, the two representations incur normalized action-selection regret and . Finally, the same gap yields a semidefinite program for sequence-aware behavior design. Thus marginal normalization constrains representation geometry, while temporal behavior controls which geometry prediction rewards.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
world models, representation learning, representation identifiability, predictive learning, temporal action structure, behavior policy design, joint-embedding predictive architectures

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_N7SuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

World models learn representations by predicting the future from data collected by some behavior policy, and that policy is usually described only by how often each action occurs. Recent theory for prediction-based representation learning says when such learning recovers the true state up to a rotation, but it leaves open whether two policies that look identical under the usual coverage summaries can lead to different learned representations. This paper argues that action frequencies leave out something important: the order in which actions are combined over time can decide which representation the learning objective prefers. In both cases the representation keeps all of the state information. What changes is whether the state appears in simple linear coordinates or in a bent, nonlinear form.

The authors study an idealized world with Gaussian states and noisy linear effects of actions. There they prove that one number decides the outcome: how much the least predictable linear direction of the state beats the most predictable quadratic feature in predictability. If this gap is positive, the best representation is a rotation of the state and nearly optimal ones stay close to it. If it is negative, a slightly bent but still invertible representation scores higher. They construct two policies that agree on action frequencies, on all statistics over any three time steps, and on all linear predictive structure, yet fall on opposite sides of the boundary. They also show that the gap can be estimated from paired simulated futures and used to design a policy whose gap is positive. In a small rendered-image version of the same world, a convolutional encoder learns the predicted linear or nonlinear coordinates in every paired run. Those coordinates persist after a world-model predictor is attached and trained, and they change which action sequence a simple planner picks.

Critical strengths
  1. Action order alone can change the learned coordinates, shown with all lower-order action statistics matched. Policies are normally summarized by how often each action occurs. The paper shows that the order in which actions are combined can, by itself, decide whether prediction-based learning recovers the state in simple linear coordinates or in a nonlinear form, even though both keep all of the state information.
  2. The theory correctly predicts which coordinates a trained CNN encoder learns. The outcome is decided by the sign of a single gap, which can be estimated from paired simulated futures and used to design a policy on the favourable side.
Critical weaknesses
  1. The experiments use the theorem's own world and training objective, not standard world-model training. Every neural experiment runs in the exact two-dimensional Gaussian world. Every one also trains on two independent futures from the same reset state, an objective that needs a resettable simulator. The LeWorldModel stage starts from encoders already selected under this objective and keeps it as the main encoder loss (Table 5). §5 concedes that ordinary single-future training from scratch is untested. Theorem 1 does cover single-future training with a perfect predictor, so what is missing is evidence that practical training reaches the predicted solution. The title and abstract ("predictive objectives", "LeWorldModel") claim more than this shows.

    The two action laws are also hand-built to match many statistics, and no ordinary policy (for example, one that tends to repeat rather than alternate actions) is shown to cross the boundary. So it is unclear whether the effect arises under behavior policies used in practice.

  2. The planning experiment favours linear coordinates through the way it is scored. The planner picks the action sequence whose predicted representation is closest to the goal's representation in squared Euclidean distance. Regret is then measured by squared Euclidean distance in the true state (§4.3).

    • When the representation is a rotation of the state, the two distances are equal. Under any other invertible map they differ.
    • So the nonlinear coordinates learned under the negative-gap law must have higher regret even if they predict the future equally well. The authors themselves call this "surrogate geometry, not predictor incapacity".
    • The regret gap of 0.293 therefore re-measures that the learned coordinates are nonlinear. It does not show a separate cost to the task.
  3. The experiments section is hard to follow. Much of it records how runs were produced rather than what they show: development versus held-out seed pairs, which checkpoints were reused, and when labels were revealed (e.g. "three disclosed development pairs and then seven frozen held-out pairs, not a post-hoc subset"). It also uses several terms before defining them, such as "ordered word", "neutral fiber" and "surrogate geometry" (abstract, Secs. 3.4 and 4.3). This hides what each experiment tests, which matters because the experiments are where the paper must show that the effect reaches ordinary world-model training. The bookkeeping belongs in the appendix.

Questions for authors
  1. (Weakness 1: training objective) Suppose the encoder and the LeWorldModel predictor are trained together from random initialization, using only the standard single-future prediction loss and the same normalization. Do the two action laws still produce state coordinates and nonlinear coordinates, respectively?
    • If they do, the title claim about predictive objectives would be supported, and my main concern would largely go away.
    • If the difference disappears, the effect would be specific to the two-future objective, which needs a resettable simulator.
  2. (Weakness 1: hand-built action laws) Does any ordinary policy family with fixed action frequencies cross the zero-gap boundary, for example a Markov chain with a tunable tendency to repeat the previous action? How large is the gap when the state has more than two dimensions? If the sign flips only for laws engineered to match many statistics, the claim that behavior design matters for representation learning holds mainly in principle.
  3. (Weakness 2: planning metric) Can you report a downstream measure that does not automatically favour a representation that is a rotation of the state? Examples are planning with a cost learned in representation space, or task success under closed-loop replanning.
    • A clear gap on such a measure would show a real task cost of the nonlinear coordinates.
    • No gap would mean the planning result only restates that the learned coordinates differ.
8 comments
AuthorsOpenReview ↗

Response to Reviewer N7Su (1/3): Joint single-future learning and claim scope

Thank you for the concrete tests and for recognizing the matched-policy result. We have uploaded a revised manuscript. We answer the single-future question here, ordinary policies and dimension in our second comment, and the planning metric and presentation in our third.

Weakness 1: training objective — Joint single-future training from random initialization

Suppose the encoder and the LeWorldModel predictor are trained together from random initialization, using only the standard single-future prediction loss and the same normalization. Do the two action laws still produce state coordinates and nonlinear coordinates, respectively?

Yes, in two of three paired initializations. We jointly train a random image CNN and the released LeWorldModel predictor core. Each example contains one starting image, five ordered actions, and one noisy future image. Training uses squared prediction error with differentiable covariance normalization, without state labels, replica loss, or a preselected encoder. The two laws share initial weights within each pair.

At 24,000 encoder updates, affine state-recovery R-squared on shared evaluation states is:

Paired initializationPositive-margin lawNegative-margin law
10.97810.4784
20.97330.4723
30.97270.9705

In the two separating pairs, quadratic features explain about 38% of negative-law representation variance, versus less than 0.4% under the positive law. The third pair recovers state under both laws. All pairs and both evaluated checkpoints are reported (Sections 4.1-4.2, Figures 2-3; Appendix N.1, Table 7). The table uses shared evaluation; Figure 2 uses the separate raw diagnostic specified in Appendix O.2.

Weakness 1: training objective — Population theory and the practical training claim

what is missing is evidence that practical training reaches the predicted solution.

The title and abstract ("predictive objectives", "LeWorldModel") claim more than this shows.

The new experiment supplies a practical discovery test; the theorem remains a population-optimum statement. The predictor core is adapted to complete five-action sequences and a two-dimensional embedding, using covariance normalization. This tests the whitening-based setting, rather than claiming the full native 192-dimensional SIGReg recipe. The Abstract and Section 4.1 now identify the adaptation, single-future training, and two-of-three outcome explicitly; Appendix O.1 gives the architecture. The result shows discovery is possible under joint single-future optimization, with initialization dependence.

AuthorsOpenReview ↗

Response to Reviewer N7Su (2/3): Ordinary policies and higher-dimensional margins

Weakness 1: hand-built action laws — An ordinary repeat-versus-switch policy

Does any ordinary policy family with fixed action frequencies cross the zero-gap boundary, for example a Markov chain with a tunable tendency to repeat the previous action?

Yes. We evaluated precisely this Markov family. The first action is uniform; each subsequent action repeats its predecessor with probability p. Every action marginal remains uniform. In two dimensions, the population margin is 0.0341 at p = 0.3 and -0.0453 at p = 0.7, with both signs certified by exact rational arithmetic (Section 3.6, Table 1; Appendix M.1).

The ordinary policy also changes low-order correlations and linear predictability. It establishes boundary crossing in a familiar policy family, while the matched five-action construction separately isolates changes beyond those summaries.

Weakness 1: hand-built action laws — Margins beyond two dimensions

How large is the gap when the state has more than two dimensions?

The crossing persists in coupled four- and eight-dimensional examples. At coupling 0.5:

State dimensionMargin at p = 0.3Margin at p = 0.7
40.0078-0.0692
80.0045-0.0789

The full coupling scan is reported in Appendix M.1, including stronger coupling where both endpoints are negative. These are population operator calculations, complementary to image training. They show that the policy and dynamics jointly determine the boundary, rather than a dimension-independent guarantee.

AuthorsOpenReview ↗

Response to Reviewer N7Su (3/3): Planning metric, task success, and readability

Weakness 2: planning metric — What the planning regret establishes

The regret gap of 0.293 therefore re-measures that the learned coordinates are nonlinear. It does not show a separate cost to the task.

We agree that the fixed-cost ranking experiment does not establish a task-independent disadvantage of nonlinear coordinates. It measures how coordinate geometry changes action ranking under a fixed Euclidean goal cost. Section 4.3 now makes this interpretation explicit: regret is 0.073 versus 0.366, with paired gap 0.293 (95% CI [0.250, 0.336]) across seven paired runs (Figure 4; Appendix N.3). Nonlinearity alone does not force worse ranking for every task.

We separately test capacity-dependent accessibility. For jointly learned single-future encoders, nonlinear readouts reduce the first two affine state-recovery gaps from 0.500 and 0.501 to 0.088 and 0.156; the third pair is linearly recoverable under both laws (Section 4.2, Figure 3; Appendix N.1, Table 7). This supports the finite-model consequence without treating readout accuracy as task success.

Weakness 2: planning metric — An independent task measure under closed-loop replanning

Can you report a downstream measure that does not automatically favour a representation that is a rotation of the state?

Examples are planning with a cost learned in representation space, or task success under closed-loop replanning.

We added the closed-loop task-success evaluation you suggested, in the public PushT simulator. Success uses the physical position-and-angle criterion, independently of latent coordinates. With 50 controls and replanning every 25 controls, results on the same 32 tasks are:

Model / collectionTerminal success
Public pretrained checkpoint13 / 32
From scratch: independent actions0 / 32
From scratch: autoregressive actions0 / 32

The public checkpoint also achieves 20 first hits. The matched scratch models use the native 192-dimensional architecture and SIGReg; the 32 tasks are evaluation cases, not training repeats. This establishes image-based task feasibility, while a matched collection-policy improvement in task success is not yet established. The public checkpoint has a different training history and cannot supply that causal comparison (Section 4.5; Appendix N.4 and O.5).

7. Experimental organization and terminology

Much of it records how runs were produced rather than what they show:

It also uses several terms before defining them, such as "ordered word", "neutral fiber" and "surrogate geometry"

We rewrote Section 4 around the scientific comparisons: joint learning, state accessibility, action choice, behavior design, and closed-loop evaluation. The Introduction defines an action word as an ordered action sequence. The abstract and experimental narrative use direct descriptions instead of the cited shorthand; model and training details are consolidated in Appendix O. Figures 2-5 present the corresponding findings and comparisons. Thank you for identifying what made the experimental argument difficult to follow.

Reviewer_gvMpOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this paper, the authors come up with a work under ideal settings to study how temporal order of actions can affect the latent representations in a predictive manner (i.e., world models). The proposed work is based on LeJEPA where it enforces all latent states to follow a standard Gaussian. Under this setting, they define a spectral gap between predictable linear state features and predictable quadratic features. They derive theoretical analysis to prove that such a positive gap will tend to true state coordinates, while a negative one will yield invertible nonlinear latent states. They also introduce metics to estimate and optimize such gap. They conduct experiments with different CNN encoders in a controlled, self-designed and toy-based manner.

Critical strengths
  1. The research question is interesting. From my understanding, the authors ask "how can we tell whether its action sequences cause a bad representation, and what should we change to improve the learned model".

  2. To evaluate the feasibility of the proposed spectral gap, the authors conduct both theoretical analysis and empirical studies (even though the experiments are very controlled without significant realistic meaning).

Critical weaknesses
  1. I find this paper very hard to follow. It reads like an auto-research generated fully by Claude (especially its figures).

  2. It seems like the proposed research is based on LeJEPA, however, its connection to such world models is clear. In LeJEPA, the training of encoders should be self-supervised. In this work, all experiments train their CNNs directly with the proposed replicated-future objective, rather than a typical single-future end-to-end objective. So I have no idea how to read this proposed work. Is it a JEPA? If not, then what is point of such a spectral gap under a very-controlled setting with toy experiments.

  3. In LeJEPA, the encoder is not trained in a supervised manner from scratch. In this work, the predictor is added after the encoder has been selected by the paper’s objective, so the study does not show that standard training independently discovers the claimed representations.

  4. My main concern is the experiments. The experiments are designed in a very controlled way with handcrafted CNN, low-dimensional Gaussian state and specially designed short action sequences. There is no benchmark datasets included or conducting more realistic experiments in a VLA system.

  5. The key motivation of this paper is to show two action sequence distributions can yield different latent states. However, the authors mentioned nothing about when a practitioner with one behavior policy should change its action sequences or how to do so in a real system.

  6. The empirical studies are poorly performed. The action-selection experiment is a small and one-word open-loop test. It does not evaluate closed-loop planning or demonstrate improved performance on a realistic world-model task.

Questions for authors

See my concerns in Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer gvMp (1/4): AI use, figures, and research provenance

Thank you for recognizing the research question. We have uploaded a revised manuscript. We address your AI-use and provenance concern explicitly here; the following comments answer the scientific concerns individually.

1. AI assistance, figures, and scientific verification

It reads like an auto-research generated fully by Claude (especially its figures).

Generative AI assisted with research ideation, mathematical discussion, Lean development and debugging, and manuscript polishing; this involvement is disclosed. The authors reviewed the AI-assisted content and take full responsibility for the final claims, proofs, code, figures, and research materials (AI use statement, p. 10).

The reported observations and scores come from simulations, training runs, and recorded evaluations. In the controlled studies, specified stochastic dynamics generate states and transition noise, and a renderer produces images. PushT observations and success labels come from the public simulator. Numerical plots are generated from computed quantities and saved evaluation outputs.

MaterialVerification and manuscript location
Spectral criterion and matched lawsDerivations and exact rational construction (Sections 3.3-3.4; Appendices B, I)
Learning and task resultsTraining/evaluation outputs and full paired results (Section 4; Appendices N-O)
Formalized componentsLean declarations, premises, and coverage map (Appendix P)

Verification rests on the derivations, executable tests and certificates, and original outputs listed above. The Lean checks cover the listed components and conditional statements, not every analytic step. These derivations, executable methods, and measured results provide the basis for the scientific claims. We revised the figures and exposition to make that evidence easier to inspect.

AuthorsOpenReview ↗

Response to Reviewer gvMp (2/4): JEPA, self-supervision, and joint discovery

2. Connection to JEPA and the normalization assumption

The proposed work is based on LeJEPA where it enforces all latent states to follow a standard Gaussian.

Is it a JEPA?

Our criterion analyzes normalized predictive representations; it does not assume exactly Gaussian encoder outputs. Gaussianity concerns the controlled state-transition model, while encoders are centered and covariance-normalized. The practical single-future experiment is a JEPA-style image-embedding predictor using the released LeWorldModel core, with the adaptation stated explicitly.

For an unrestricted action-conditioned predictor, optimal single-future squared error equals the representation dimension minus encoded-state predictable energy. Theorem 1 applies to this score as well as the replica score (Sections 3.1-3.3). Replicas are an analytical and estimation route; the single-future experiment below does not use them.

3. Self-supervision and single-future data

all experiments train their CNNs directly with the proposed replicated-future objective, rather than a typical single-future end-to-end objective.

We added joint single-future training without state supervision. Each example has a starting image, five ordered actions, and one noisy future image. Squared prediction error and differentiable covariance normalization train both networks. No replica loss or state label enters training. The replica study is also label-free, but requires conditional future pairs; it is now distinguished from the new discovery experiment (Section 4.1; Appendices N.1 and O.1).

4. Independent discovery from random initialization

the predictor is added after the encoder has been selected by the paper’s objective, so the study does not show that standard training independently discovers the claimed representations.

Both encoder and predictor now start from random initialization and learn together. Two of three paired initializations discover the predicted shift. Shared-evaluation affine state-recovery R-squared is:

Paired initializationPositive-margin lawNegative-margin law
10.97810.4784
20.97330.4723
30.97270.9705

The third pair recovers state under both laws. In the separating pairs, quadratic features explain about 38% of negative-law variance versus less than 0.4% under the positive law. Wider nonlinear readouts reduce the first two affine recovery gaps from 0.500/0.501 to 0.088/0.156 (Sections 4.1-4.2, Figures 2-3; Appendix N.1, Table 7).

The released predictor core is adapted to the full five-action sequence and two-dimensional embedding, with covariance normalization (Appendix O.1). This supplies joint discovery evidence without a preselected encoder. Native 192-dimensional SIGReg models are evaluated separately in PushT.

AuthorsOpenReview ↗

Response to Reviewer gvMp (3/4): Controlled evidence and behavior design

5. Why the controlled construction, and what extends beyond it?

The experiments are designed in a very controlled way with handcrafted CNN, low-dimensional Gaussian state and specially designed short action sequences.

The matched construction isolates an effect that coverage and linear predictability cannot detect. It fixes dynamics, state distribution, renderer, all statistics on up to three action positions, and the full linear predictable-energy operator. Yet the two margins are above 0.02 and below -0.02, respectively (Section 3.4, Proposition 2). This is a mechanism-isolating construction, rather than a benchmark comparison.

We also added ordinary repeat-versus-switch policies and coupled higher-dimensional examples. With uniform action marginals, population margins are:

State dimensionMargin at p = 0.3Margin at p = 0.7
20.0341-0.0453
40.0078-0.0692
80.0045-0.0789

The 4D/8D examples use coupling 0.5; the full scan includes stronger coupling where both endpoints are negative. These are population calculations, distinct from the joint image-learning test in our second comment (Section 3.6, Table 1; Appendix M.1). The ordinary policies change low-order correlations too; the matched pair identifies the additional higher-order effect.

6. When and how should a practitioner change behavior?

when a practitioner with one behavior policy should change its action sequences or how to do so in a real system.

Within the model assumptions, the margin supplies a diagnostic and a design objective. Starting from one law, estimate linear and quadratic predictable energy using states or calibrated features and resettable rollouts. A negative margin means linear coordinates admit an improving nonlinear perturbation. To favor linear recovery, compare candidate sequence laws and maximize the margin subject to action-coverage constraints.

Section 4.4, Figure 5 demonstrates this procedure under matched action moments through degree three. Across 20 pilot draws with 4,096 replica pairs per sequence:

Behavior-design resultMean [95% CI]
True margin of selected law0.02468 [0.02432, 0.02504]
Gain over linear-predictability-only design0.00812 [0.00061, 0.01563]

This is an actionable rule when those features and conditional rollouts are available (Appendix O.4). Applying it directly to arbitrary offline pixel logs additionally requires identifying the relevant features and operators; the PushT comparison does not establish that transfer.

Reviewer_FpTwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors consider state representation learning from state-action trajectories. The provide, for the population linear Gaussian setting, sufficient conditions under which a non-linear state encoder recovers a linear transformations of the true states. Then then show that linear recovery can depend on the policy under which the data was collected even when the dynamical model and the marginal action distributions remains fixed. Lastly they show that there are cases (when the downstream task is linear) where a linear state representation is preferable.

Critical strengths
  • The paper is, in general, well written and visualizations are useful to better understand the core idea of the paper.
  • I think the results that under a L2-projection objective different behavioral policy can imply different learned representations is quite interesting.
Critical weaknesses

I am struggling a little with the motivation of the paper. It seems to be assumed throughout that a state-representation that is linear in the true state is preferable but not why this is the case. They show at the end that there are cases where a state-representation that is linear in the true state can perform better for downstream tasks if the true dynamics are linear, however, I am hypothesizing that there are other settings where to opposite holds. Therefore I don't understand what statement this result is supposed to support, besides that for different downstream tasks one might want to prefer one or the other state representation.

More generally, the results are somewhat limited as it assumes state independent policies are used.

Questions for authors

Since state representation is not a compression here it seems to be that all bijections of the "original state" are "equally good" representations. The inductive bias of the objective to choose a representation that is predictive seems reasonable to me. Can the authors explain in what sense a linear representation is preferred. Also if we know the dynamics to be linear why don't we learn the representation directly using this knowledge?

Can the results me generalized state depended policies?

2 comments
AuthorsOpenReview ↗

Response to Reviewer FpTw (1/2): Motivation, bijections, and model capacity

Thank you for recognizing the interest of policy-dependent representation learning. We have uploaded a revised manuscript. We address your motivation and information-preservation questions together below; our second comment answers the state-dependent-policy question.

1. Why should coordinate choice matter?

a state-representation that is linear in the true state is preferable but not why this is the case.

all bijections of the "original state" are "equally good" representations.

They are equivalent in retained information, but not necessarily in accessibility to a fixed model class. An unrestricted model with a correspondingly transformed task cost can use either representation. A limited predictor or readout may require more capacity after a nonlinear coordinate change. Our claim concerns that interaction; linear coordinates need not be best for every task.

Section 3.5, Proposition 3 makes this distinction explicit. The same smooth, invertible transformation preserves all state information while producing:

Predictable-energy scoreChange from state coordinates
Unrestricted predictor+0.00882
Best action-conditioned linear predictor-0.03357 (14.9% of baseline)

The map and its inverse both have Lipschitz constants below three. Thus the richer predictive objective can favor well-conditioned coordinates that a linear model finds harder to use.

The new readout experiment tests this empirically. After joint single-future training from random initialization, the positive-law minus negative-law state-recovery gaps decrease with nonlinear readout capacity:

Paired initializationAffine R-squared gapMLP-128 gap
10.5000.088
20.5010.156
30.002-0.001

Both readouts use shared evaluation states and matched fitting conditions; gaps are computed before rounding. The third pair recovers state under both laws. These results support a capacity-dependent interpretation of coordinate accessibility (Section 4.1, Figure 2; Appendix O.1, Table 7; methods P.2).

2. Why not use known linear dynamics?

if we know the dynamics to be linear why don't we learn the representation directly using this knowledge?

Using that prior is sensible when available. We ask what prediction selects when latent linearity is not imposed. Theorems 1 and 2 show that temporal behavior can reverse the objective's coordinate preference even when all statistics on up to three action positions and the complete linear predictability operator agree. This identifies a property of data collection that action-frequency summaries miss. We now state this motivation explicitly in the Introduction and Sections 3.2-3.5.

AuthorsOpenReview ↗

Response to Reviewer FpTw (2/2): State-dependent policies

3. Extension to state-dependent policies

the results are somewhat limited as it assumes state independent policies are used.

Can the results me generalized state depended policies?

Yes. For an explicit family of state-dependent policies, the same spectral criterion gives both state recovery and nonlinear improvement. For centered, covariance-normalized encoders with the same dimension as the state, the result is:

Spectral marginResult under state feedback
PositiveEvery optimal encoder is a rotation of the state. Mean squared distance to a state rotation is at most twice the score deficit divided by the margin.
NegativeEvery state rotation admits arbitrarily close, smooth, invertible nonlinear changes that strictly improve prediction.

A concrete state-dependent policy: when opposite signed channel pairs are available, choose the positive sign if the absolute first state coordinate is below a threshold, and the negative sign otherwise. This genuinely changes the selected action with the current state. Each channel pair retains its fixed total probability; the complete channel is selected at the start of the step.

Why the result holds: fixed pair totals and sign probabilities unchanged by state reversal preserve the standard Gaussian state marginal and every full-state predictable-energy operator block. The spectral margin is therefore identical to that of the corresponding state-independent channel mixture. Invertible encoders also retain the equality between full-state and encoded-state predictive scores, giving the negative-margin conclusion above.

The construction, operator identity, and proof are given in Appendix K.2, Proposition 22, together with the recovery and stability bound. This establishes the extension for the stated state-feedback family.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Further evidence for joint learning and physical downstream use

Thank you for your comments on joint learning and physical relevance. We have uploaded a further revised manuscript. This update supplements our earlier replies (2/4), on self-supervision and joint discovery, and (4/4), points 7–8, on public tasks and closed-loop performance.

1. Single-future training and independent discovery — Critical Weaknesses

“all experiments train their CNNs directly with the proposed replicated-future objective”

“the predictor is added after the encoder has been selected by the paper’s objective”

We now jointly train the released approximately 18M-parameter LeWorldModel architecture from random initialization, using its native 192-dimensional embedding, single-future prediction and SIGReg. There is no hard whitening or state-supervised training target. Images come from a 192-dimensional controlled world; ordinary Markov policies preserve one-action marginals while changing temporal dependence.

Linear feature content is the share of raw representation variance associated with linear state features.

Measurement across three paired initializationsp = 0.3p = 0.9Paired difference [95% CI]
Linear feature content52.9%38.4%14.55 percentage points [6.90, 22.21]

All three pairs have the same ordering. All models outperform mean and persistence prediction baselines and respond to action shuffling. This directly tests feature differences discovered during joint learning, with the released architecture at native width (Section 4.2, Figure 3; Appendix N.2).

2. Public data and physical performance — Critical Weaknesses; follow-up to our reply (4/4), points 7–8

“There is no benchmark datasets included or conducting more realistic experiments in a VLA system.”

“It does not evaluate closed-loop planning or demonstrate improved performance on a realistic world-model task.”

We add official PushT demonstrations and three physical comparisons using the same six policy-adapted encoders, from three paired adaptations of the public pretrained model:

Physical task measureSwitchRepeatMean change
Block-position squared error (px²), lower is better88.9277.8612.4% lower
Forecast-based action-choice regret (px), lower is better13.5912.0711.2% lower
Closed-loop native score, higher is better0.6510.6896.0% higher
Closed-loop strict success30.3%31.7%+1.4 percentage points

Position estimation uses 96 episodes held out from readout fitting/selection; forecast-based choice uses 96 fresh initial states. Feedback control uses 48 fresh shared initial states and three matched controller fits per encoder. All 18 adapted visual controllers achieve 10–23 strict successes out of 48 tasks. Position and score comparisons, after averaging their three fitted heads, favor repeat in all three adaptation pairs; the forecast comparison also has the same direction in all three pairs. Repeated heads are not additional world models.

Absolute gainAcross-adaptation 95% CI
Position-error reduction (px²)[-3.31, 25.42]
Action-choice regret reduction (px)[-1.18, 4.24]
Native-score increase[-0.0337, 0.1113]

These are new evaluations, separate from the earlier 32-task scratch-trained planning comparison. The closed-loop result uses frozen adapted visual features and demonstration-trained feedback controllers; forecasts drive the separate action-choice task. We report both the task differences and their training uncertainty.

Section 4.3, Figure 4 and Appendices N.3–N.5 give all paired comparisons and controls; methods are in O.4–O.6. The earlier planning study is retained in N.7. Following point 9 of our earlier reply, the manuscript now connects state estimation, forecast-based choice and feedback control in one experimental narrative, with methods collected separately.

AuthorsOpenReview ↗

General response and summary of revisions

We thank the reviewers and the Area Chair for their careful reading and constructive feedback. Our central contribution is to characterize how temporal action composition changes the coordinates favored by prediction, even when action coverage and the complete linear prediction operator agree. The revisions connect this theoretical mechanism to joint learning and downstream use. We summarize the changes by reviewer below.

Reviewer concernMain revision or resultManuscript location
FpTw: motivation, bijections, and the value of state coordinatesWe distinguish information preservation from accessibility to a finite learner. The analysis and matched affine/nonlinear readouts show why equally informative coordinates can differ in use. We also explain the role of a known linear-dynamics prior.Introduction; Section 3.5; Section 4.1, Figure 2
FpTw: state-dependent policies; N7Su: ordinary policy familiesAn explicit state-dependent policy family preserves the spectral criterion. Ordinary repeat-versus-switch Markov policies also cross the theoretical boundary.Appendices K.1–K.2
N7Su W1/Q1; gvMp: joint single-future learning at native widthWe train the released approximately 18M-parameter LeWorldModel from random initialization with its native 192-dimensional embedding, single-future prediction and SIGReg, without hard whitening. All three pairs show the same ordering: 52.9% versus 38.4% linear feature content, with a paired difference of 14.55 percentage points [95% CI: 6.90, 22.21].Section 4.2, Figure 3; Appendix O.2
N7Su W2/Q3; gvMp: public tasks and physical downstream consequencesOn public PushT data, all six adaptation pairs favor repetition in matched position estimation: 88.93 to 78.38 px² (11.9% lower), with paired error-reduction interval [6.64, 14.46] px². Without refitting, the same models and readouts retain 10.4% lower error on 256 fresh states (318.23 to 285.00 px²), again in all six pairs, with interval [15.20, 51.27] px².Section 4.3, Figure 4a–b; Appendix O.3, Tables 10–12; methods P.4

The six physical adaptation pairs share one public pretrained starting model. The intervals above describe variation across adaptations on each shared evaluation panel. Forecast-based action selection and successful visual feedback control are evaluated separately, with their own training uncertainty in Figure 4c–d and Appendices O.4–O.5.

Together, these results connect the theoretical coordinate-selection mechanism to native-width learning and reproducible differences in physical-state accessibility under matched downstream capacity. Our reviewer-specific replies address the individual questions in detail. We appreciate the reviewers' continued consideration and welcome any remaining questions.

For ease of reference, the reorganized appendix places wider-representation and finite-predictor analysis in D, behavior-policy extensions in K, experimental results in O, methods in P, and Lean verification in Q.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.