acceptodds
Under review as a conference paper at ICLR 2027

SpikeWorld: Adapting Frozen World Models with Compact State

Abstract

A world model can predict the future yet remain unable to absorb the errors revealed when that future arrives. We ask how much mutable state is needed to adapt its predictions without rewriting its learned representation. We introduce SpikeWorld, which pairs a next-event predictive–semantic model with a bounded external fast state. We separate the shared computation learned offline from the residual corrections written online: completed actions supervise later predictions, while a distinct score state calibrates registered action transformations. On a continual stream of registered actuator changes, 29 KB of mutable state obtains a 19.3% prequential prediction gain, versus 21.9% for prediction-path tuning with 67× more mutable state. Controlled formation tests establish temporal predictive structure, and a same-stream routed matrix RLS comparison provides a compact actuator-identification reference. SpikeWorld makes deployment plasticity explicit: retain the predictive representation, and allocate bounded state to what new observations teach.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
world models, spiking neural networks, test-time adaptation, fast-state adaptation, predictive representations, online learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1xiGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies world models that can update their internal state from information encountered online, without weight updates. To this end, the authors introduce SpikeWorld, a multimodal world model based on spiking neural networks. It maintains an external memory that is written to and read from at deployment time. The authors report clear improvements in prediction accuracy over the frozen base model when using this deployment-time memory. They also report prediction metrics and closed-loop control performance that match weight fine-tuning.

Critical strengths
  • Interesting research question. How to use information obtained online to adapt a world model without weight updates is an interesting and timely research question.
Critical weaknesses
  • Dense and often incomprehensible writing. Parts of the paper are extremely dense and technical and lack sufficient explanation. The following examples all come from a single paragraph of the introduction:

“Sensory streams predict the next spike-bearing slice”: spikes and “spike-bearing” have not been introduced at this point. Spike Transformer is mentioned in the preceding sentence, but no citation or explanation is given. Spiking neural networks, and why they are relevant here, should be introduced in a sentence or two. “action streams predict a later physical endpoint”: the paper does not explain what a “physical endpoint” is. “Controlled tests show that this predictive interface captures temporal organization beyond rate and order controls, predicts label-free future statistics better than random state, and supports bounded cross-modal composition”: the “rate and order controls” are not described, and it is unclear what “label-free future statistics” are. “Cross-modal composition” can be guessed, but “bounded” is again unclear in this context.

Similarly dense, cluttered sentences appear in several sections of the paper, and acronyms are used before they are introduced (e.g., “routed matrix RLS comparison”). This makes the paper very hard to parse and, in places, incomprehensible.

  • Key parts of the method are underexplained. The method description is not detailed enough to judge the approach. Losses, key concepts and terminology are not properly introduced. Vague descriptions such as “the semantic head reads the present state” leave it unclear what the “state” of this model is. Is it, for example, a text description? In many places the reader is left to guess and fill in the gaps. As a result, I could not follow what the model actually does, and so I cannot verify whether the reported results support the paper’s claims. This is the main reason for my soundness and informativeness score.
Questions for authors
  • Missing key motivation. Why is a spiking neural network the right tool for the task the authors consider?
  • Unjustified constants in equations. Eqs. 3, 7 and 8 contain a factor of 1/39. It appears to be a common hyperparameter across these equations, but it is never introduced. Eq. 4 contains factors of 0.25 and 4. What is the justification or reasoning for these values?
  • Undefined symbols. The loss in Eq. 4 contains LsemL_\text{sem}, LnextL_\text{next} and LbindL_\text{bind}, but none of these terms is introduced. What does each of them do? What are WoutW_\mathrm{out} and CptC_{pt} in Eq. 6? Short natural-language descriptions of all quantities are crucial for understanding the method.
  • Undefined key concepts. What are shear values? What are attenuation values?
  • Byte counts. In several places, the paper computes and reports the size of matrices in bytes but does not explain why this is relevant. What does the byte count tell the reader?

Suggestions:

  • Update low-quality figures. In Figures 1 and 2, labels overlap arrows and box outlines, and some labels extend beyond their boxes. Math symbols (e.g., h_t) are not typeset in math mode. Instead, both math and regular text use a Comic Sans-style font. Figure 1 in particular is not referenced in the text. What is it meant to contribute to the paper?
Reviewer_ogcGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper considers the problem of modeling dynamics of a system that requires online adaptation. In particular, it considers the case of adapting a dynamics ("world") model with frozen weights using a small set of parameters ("mutable state"), which needs to adapt to a continual stream of actuator changes. Specifically, those changes (which are per-time-step transformations of the effect of the actions, e.g. noise or artificially lowered gains), are registered in advance in a candidate bank which also has fixes for the changes (the inverse), as in multiple-model adaptive control.

The method is tested on Meta-World robotic trajectory prediction across four tasks.

Critical strengths
  1. Comprehensive experimental protocols with detailed comparisons across multiple arms, carefully ablating which aspects of the model cause the improvement.
Critical weaknesses
  1. The choice of a spiking transformer as the model architecture was not justified, and the main results doesn't seem to depend on the spiking design.
  2. The setting is scoring against a fixed bank of known faults with known inverses. Significant cited literature (e.g. Kumar et al., 2021) demonstrate adaptation to high variability of continuous variables. Overall scale of the results are extremely small, both in terms of number of parameters and the number of tasks.

Potentially minor: It may be that I'm coming primarily from a machine learning background (as opposed to e.g. neuroscience), but the terminology used in this paper seems quite non-standard for an ML conference. For example, "formation" instead of "pretraining", "deployment plasticity" instead of "online adaptation", "implementation closure", etc.

Questions for authors
  1. Could you expand on the use of the spiking transformer, as it seems core enough to the idea that the method is named SpikeWorld? Ideally, it would be very helpful to have a non-spiking transformer as a baseline.
  2. How does the method benchmark to standard PEFT methods? I would recommend comparing with standard Parameter-Efficient Fine-Tuning methods like LoRA. In particular, a LoRA of the same rank on the same parameters being updated online with SGD instead of Adam. A huge part of the memory gap seems to come from the optimizer being Adam.
Reviewer_13PXOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes SpikeWorld, which aims to combine next-event predictive semantics with a bounded external fast state. However, I found the paper difficult to follow because many symbols and constants are introduced without sufficient explanation.

Critical strengths
  1. The problem addressed by the paper seems important.
Critical weaknesses
  1. The presentation is difficult to follow. Many symbols and constants are introduced without explanation; for example, the meaning and role of (1/391/39) are unclear. The reported intervals are difficult to parse visually in the text, and the diagrams contain too much text and appear to contain inaccuracies.

  2. The writing requires substantial revision to improve clarity. In its current form, the lack of clear definitions and explanations makes it difficult to understand the proposed method and assess its contribution.

Questions for authors

Please revise the paper to improve its readability. In particular, define symbols and constants when they are first introduced, explain the meaning and role of constants, improve the formatting of the reported intervals, and revise the diagrams for clarity and accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.