acceptodds
Under review as a conference paper at ICLR 2027

TurboSens: An Interactive Turbofan Environment for Probing World Model Representations against Hidden Engine State

Abstract

World models are expected to learn compact representations that capture the underlying state of dynamical systems, but evaluating whether these representations preserve meaningful latent state remains difficult under partial observability. We introduce TurboSens, a controlled environment for measuring latent state preservation in world models. TurboSens provides a configurable turbofan simulation with paired latent state and sensor observations, allowing representation quality to be evaluated against known underlying system state rather than only through prediction or reconstruction performance. The environment is designed to simulate issues that could arise when relevant state is only partially observable and must be inferred from temporal context, actions, and changing operating conditions. We evaluate representations from several world model families using frozen encoders and inverse probes that measure latent state recovery, recovery as a function of observability, robustness to distribution shift, and responses to maintenance interventions. Across controlled changes in observability and degradation context, we find that self supervised representations preserve far less of the state than the observations demonstrably contain, with recoverability depending on the pretraining objective and on memory aligned with maintenance events rather than on context length. These results highlight a gap between what the observations contain and what learned representations preserve, and demonstrate how controlled simulation can provide a direct measurement of whether world model representations retain the underlying state required for prediction and decision making under partial observability.

Primary area:
datasets and benchmarks
Keywords:
World Models, Turbofan, SSL, Representations, Environment, Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XeGqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces TurboSens, a controlled environment for measuring how well world models preserve latent state. The environment is a configurable turbofan simulation, built on the OpenDeckSMR simulator and combines sensor observations with the known engine health state variables. This allows representation quality to be evaluated against the true system state, beyond prediction or reconstruction performance. The environment is designed so that part of the state is only partially observable, e.g., fouling channel, and must be inferred from temporal context, maintenance actions and changing operating conditions. The paper also releases the data produced which contain 500 training episodes.

The authors also evaluate representations from several world model families (e.g., JEPA, LSTM, RSSM model) using frozen encoders and linear inverse probes measuring the ability for latent state recovery, the robustness to distribution shift and the responses to maintenance interventions.

The authors find that self-supervised representations preserve far less of the state than the observations contain. The best self-supervised encoder reaches a mean correlation of 0.73, while a linear probe on the raw observation reaches 0.93.

Critical strengths
  • The pairing of the ground-truth state with an interactive simulator is a great contribution. The environment exposes the full health and fouling state at every step, accepts actions online and replays episodes exactly under counterfactual changes, which enables measurements that existing turbofan benchmarks cannot support.

  • The evaluation protocols are quite detailed. The reference baselines, the episode-level bootstrap, multi-seed reporting and the analysis by time since the last wash are reported in detailed.

Critical weaknesses
  • The environment does not appear to demonstrate the partial observability challenge that motivates the paper. A linear fit on a single raw observation already recovers the wear state at 0.93, an MLP reaches 0.96, and the MLP's recovery drop is very minimal (only 0.027) as fouling builds up. Thus, the claims about action-dependent observability therefore is a very small delta.

  • The conclusion about self-supervised objectives is broader than what is supported by the presented evidence. All JEPA and LSTM models share a single training recipe and the RSSM collapses even though its reconstruction loss converges, which suggests an implementation or normalization problem. The paper also attributes the gap between the self-supervised encoders (0.73) and the raw observation (0.93) to the training objective, but there is a lack of comparison against simple baselines which should isolate the if the objective formulation is the actual cause.

Questions for authors
  1. The paper argues that a single observation is consistent with many combinations of wear and fouling, so that only a model integrating the action history can recover the wear state. The reference baselines appear to suggest otherwise. A linear ridge on one raw observation already reaches 0.93, an MLP reaches 0.96, and the MLP's recovery drops by only 0.027 as fouling builds up. Could you explain why TurboSens should still be considered partially observable in a meaningful sense? For example, are there state components, wash strata or episodes in your existing results where single-frame recovery fails and the history is needed? This explanation would address my main concern about the benchmark.

  2. Section 5.1 attributes the gap between the best self-supervised encoder and the raw-observation ridge to the self-supervised objective formulation. The matched oracle rules out encoder capacity, but the experiments do not rule out that the training itself changes the encoder very little. Can evidence supporting this be provided? e.g., if the randomly initialized encoder scores far below 0.73 then the training clearly shapes the representation and would isolate the contributions to the objective as the paper argues.

  3. The RSSM model only achieves a correlation of 0.045 although its reconstruction loss converges. The paper reads this as evidence that reconstruction quality does not imply state preservation. But, on the other hand, a linear map of the observation already recovers the state at 0.93, so a model that reconstructs the observation well should retain most of this information. This makes the RSSM result unconvincing, particularly since it is based on a single seed. Could you report the RSSM's reconstruction quality on the test split, a linear and an MLP probe on its encoder embedding, and the input normalization used?

1 comment

We thank the reviewer for the precise review. We added the random initialization control proposed.

W1 / Q1 (partial observability). The small drop you point out (0.027) is due to a few sensors that reveal fouling. Without them, single frame recovery falls to 0.889 or 0.779 and history adds +0.060 to +0.067 R2R^2. The apparent failures in the oldest time since wash strata came from very low label variance and disappear after refitting, so we do not count them.

W2 / Q2 (role of the objective). Untrained encoders with the same architecture match the pretrained ones, while training the same network on the state reaches 0.911. The pretraining objectives are therefore the limiting factor, and the revision states this for the objectives and recipe we tested only.

Q3 (RSSM). You are right that this was an implementation issue. A setting in the KL term let the latent ignore the observations, and both the linear (0.045) and MLP (R2≈0R^2 \approx 0) probes confirm that it holds no state. Inputs were symlog transformed and layer normalised, and we will report reconstruction error with the corrected run.

Reviewer_roEgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents TurboSens,an interactive turbofan simulator that pairs sensor readings with ground-truth engine health, providing a testbed for what world-model representations retain. Probing frozen encoders under distribution shifts and maintenance interventions, the paper shows that the tested JEPA, LSTM, and RSSM models capture much less hidden-state information than supervised models on raw observations. Storing maintenance-event representations helps more reliably than extending context. The work is mainly a benchmark and evaluation protocol, and its conclusions are limited to synthetic dynamics and the evaluated models.

Critical strengths

A useful benchmark for directly evaluating state recovery. TurboSens pairs sensor observations with ground-truth engine states and supports controlled maintenance interventions and distribution shifts, so representations can be evaluated against the true state rather than through prediction or reconstruction loss. Comparing raw-observation baselines and a supervised matched-backbone model using nonlinear-probing reveals a gap between what the observations contain and what the learned representations retain.

Critical weaknesses
  1. The temporal-memory challenge is not clearly established. A single-observation MLP recovers the state with 0.963 correlation, and ridge regression on the raw observation reaches 0.930. Recovering the hidden state therefore does not seem to require maintenance history. The experiments mainly show shortcomings of the learned representations rather than confirming the benchmark's central difficulty.

  2. The evidence does not isolate why the representations falls. Low probe recovery does not by itself establish information loss. The matched-backbone oracle also cannot separate limitations of the pretraining objective from optimization difficulties. Stronger training controls are needed to support the paper's broader conclusions about world-model objectives given the following results:

  • Seed variance is large within a single configuration. (0.655 to 0.770)
  • Most SSL encoders score below the sensor-mean baseline (0.694).
  • The near-collapsed RSSM score (0.045, one seed) further weakens comparisons across model families.
Questions for authors
  1. Many interactive simulators already expose ground-truth state: MuJoCo and the DeepMind Control Suite have fully specified dynamics with modifiable physical parameters, and linear probing of learned representations against such state is an established protocol on tasks like Atari [1]. What does TurboSens offer that these environments do not, and why is a turbofan engine necessary to provide it? If the key property is action-dependent observability, could it be built into an existing simulator?

  2. Do the authors expect the conclusions, such as event-aligned memory outperforming longer context, to hold for world models trained in these other settings, given that a turbofan engine is a very specific setting?

  3. A two-layer MLP probe improves on the ridge probe by only 0.04 to 0.08 (Appendix E.4), and the paper takes this as evidence that the missing state is absent from the embeddings rather than hard to read out. Why do the authors consider this probe a reliable test of what the embeddings contain? A fixed-capacity probe only gives a lower bound on the information present [2,3], as the Limitations section acknowledges.

References

[1] G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes. arXiv:1610.01644, 2016.

[2] J. Hewitt and P. Liang. Designing and interpreting probes with control tasks. EMNLP-IJCNLP, 2019.

[3] T. Pimentel, J. Valvoda, R. Hall Maudslay, R. Zmigrod, A. Williams, and R. Cotterell. Information-theoretic probing for linguistic structure. ACL, 2020.

1 comment

Thank you for the careful review and for the references on probing, which we used for new controls.

W1 (temporal memory). You are right for the default sensor set. Removing the few sensors that reveal fouling makes history necessary (+0.060 to +0.067 R2R^2), and the revised paper states the claim for that setting.

W2 (why representations fall short). Untrained encoders with the same architecture recover the state about as well as the pretrained ones, while training the same network on the state reaches 0.911, so the pretraining objective is the limiting factor. The seed spread (0.655 to 0.770) stays well below the raw sensors (0.930) and does not change this. The RSSM result comes from the KL setting we used (general response), under which RSSMs are known to let the latent stop using the observations, so we no longer use it to compare model families.

Q1 (why a turbofan). Turbofans are a standard reference system in prognostics, so the setting and its physics are well documented. TurboSens adds what static turbofan datasets and control suites lack, namely a slow hidden state masked by faster effects, actions that change what the sensors reveal, all alongside the ground truth at every step.

Q2 (other settings). We no longer make the claim on event aligned memory. We expect the main finding to hold wherever the quantity of interest changes much more slowly than the observations, which TurboSens lets one test by varying the sensor set.

Q3 (probe reliability). Following Hewitt and Liang, we added a shuffled label control, which stays near zero for probes of up to four layers, so the probes do not invent structure. Even with four layers, the best pretrained encoder reaches R2=0.620R^2 = 0.620, against 0.866 for the raw sensors.

Reviewer_pT52OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TurboSens, an interactive turbofan simulation environment for evaluating whether world model representations preserve underlying health states. It provides paired sensor observations and ground-truth states under mechanical degradation, reversible fouling, and maintenance interventions. The authors evaluate JEPA, LSTM, and RSSM representations through frozen probes, distribution shifts, and counterfactual interventions. Results reveal a substantial state-recovery gap relative to supervised baselines, while memory anchored to washing events mitigates recovery degradation more effectively than simply extending temporal context in the tested configurations.

Critical strengths
  1. The paper addresses a meaningful evaluation problem by providing paired sensor observations and ground-truth health states, allowing direct assessment of what world model representations preserve beyond prediction or reconstruction performance.

  2. The interactive environment supports controlled maintenance interventions and counterfactual replay.

Critical weaknesses
  1. Lack of real-world validation. All experiments are conducted in simulation, without validation on real-world engine data or physical systems. It therefore remains unclear whether the observed representation failures and benefits of maintenance-aligned memory also hold in real-world settings.

  2. Unclear necessity of temporal reasoning. The benchmark is motivated by action-dependent partial observability, suggesting that historical observations and maintenance actions are important for recovering mechanical states. However, an MLP using only the current observation achieves a mean Pearson correlation of 0.963, while raw-observation ridge regression reaches 0.930. The authors should clarify which aspects of the state genuinely require temporal information and demonstrate conditions where history provides substantial gains over instantaneous observations.

Questions for authors
  1. Can the authors provide evidence that the findings extend to real-world settings, through real engine data?
  2. Given that a single-observation MLP achieves a mean Pearson correlation of 0.963, which state components or operating regimes genuinely require historical observations and maintenance actions?
  3. Does the benefit of wash-aligned memory specifically come from alignment with informative maintenance events?

I would consider raising my rating if these concerns are satisfactorily addressed with convincing evidence and clarifications.

1 comment

We thank the reviewer for the clear questions and for indicating which evidence would change the assessment.

W1. Lack of real world validation. Our evaluation requires the hidden health state at every step, which fleet data does not record, so it cannot score a representation. This is why TurboSens is built on a component level engine model (OpenDeckSMR) with exact ground truth, and the limitations section states this scope.

W2. Unclear necessity of temporal reasoning. With all sensors, history adds only +0.025 R2R^2 in the most fouled rows, because a few sensors reveal fouling directly. Once they are removed, history adds +0.060 to +0.067, so TurboSens can produce the condition you ask for by changing the sensor set (general response).

Q1. Evidence on real engine data. Real engine data cannot be scored without the ground truth state (W1). A possible route is label free indicators, such as monotonicity between repairs, which track true state recovery in TurboSens and could be computed on fleet data, which we leave to future work.

Q2. Which components or regimes require history. History matters where fouling is high and masks the wear, especially when no sensor measures fouling directly (W2). Only windows longer than the wash cycle help (k=160k=160, not k=16k=16), so the useful information lies in the history of washes.

Q3. Does the benefit of wash aligned memory come from the alignment. No. Memory anchored at a random time matches or beats wash aligned memory in every seed and stratum (ΔR2\Delta R^2 from −0.035 to −0.065), so we retract the alignment claim.

We hope these answers address your concerns, and we would be glad to discuss any remaining point.

Comments on the paper

1 comment

We thank the reviewers for their careful reviews. We have uploaded a revised version of the paper, with the new results in Section 5.1 and Appendix E.6.

The reviews raise two main questions. Does recovering the state in TurboSens require history, and does the gap between self supervised and raw representations reflect the training objective? We ran the experiments the reviewers asked for. They confirm the central result of the paper, make both answers more precise, and lead us to correct one secondary claim.

History and the sensor set. The reviewers note that a single raw observation recovers the state well (ridge 0.930, MLP 0.963). This holds in the default configuration, because a few sensors respond strongly to fouling and let one observation separate fouling from wear. With all sensors, a history model (160 steps with actions) improves R2R^2 by only +0.025 [0.011, 0.042] on the most fouled quarter of the test rows. Once those sensors are removed, single frame recovery drops and history brings a clear gain.

Sensors usedSingle frame (Pearson)Gain from history, ΔR2\Delta R^2 [95% CI]
All seven0.929+0.025 [0.011, 0.042]
Top 2 fouling sensors removed0.889+0.067 [0.032, 0.128]
Top 4 fouling sensors removed0.779+0.060 [0.028, 0.110]

Only windows longer than the wash cycle help, a window of 16 steps never does. TurboSens therefore lets the user set how hidden the state is, from nearly observable in one frame to requiring long memory, which a static dataset cannot offer.

What self supervised training learns. The new controls strengthen the main result. A shuffled target control stays near zero for every probe, and with four layer probes the best self supervised encoder still reaches only R2=0.620R^2 = 0.620, against 0.784 for the supervised oracle and 0.866 for the raw sensors. The gap therefore lies in the representations and not in the probe. The random initialisation control suggested by Reviewer XeGq shows more. Trained JEPA and LSTM encoders recover no more of the state than random encoders with the same architecture, whereas the same backbone trained with state supervision reaches 0.911. The objectives we tested do not learn the slow state at all. This is the kind of failure TurboSens is built to reveal, and it is invisible in training losses or forecasting metrics.

Corrections. Memory anchored at the last wash is no better than memory anchored at a random time, so the gain comes from the added memory and not from its alignment with maintenance events. We retract that claim. We also traced the RSSM collapse to free bits being disabled in our configuration (posterior KL near 8e-7). We report it for completeness and will add a run with free bits enabled.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.