acceptodds
Under review as a conference paper at ICLR 2027

Associative Dreamer: Predicting Action Consequences with Associative Memory

11 views

Abstract

In partially observed environments, identical current images and actions can have different consequences because of earlier events. We study how a world model can recover and use the relevant historical evidence within a bounded online state. We introduce Associative Dreamer, which learns to write visual associations into a fixed-size fast-weight memory, retrieve evidence through autoregressive queries, and condition its action-dependent latent prior on that evidence. At known retrieval times, an auxiliary objective aligns retrieved representations with exponential moving average (EMA) targets of annotated, query-relevant past observations. Targets come from the same training run and serve only as loss supervision. We construct a benchmark of three controlled pixel tasks whose future action consequences depend on earlier evidence, and evaluate short open-loop prediction on fixed training trajectories. Within a 20,000-update budget, the aligned model achieves 99.58% and 100% mean final-window consequence accuracy in Transport and Power, compared with approximately 50% for both an ordinary recurrent model and memory trained without alignment. At 40,000 updates in Assembly, aligned and prediction-only memory average 77.01% and 75.97% balanced accuracy, respectively. Disabling future reads removes the successful models' predictive advantage. These results show that Associative Dreamer combines retrieved historical evidence with the current state and actions to predict future consequences that depend on earlier experience.

Primary area:
learning on time series and dynamical systems
Keywords:
world models, associative memory, fast weights, autoregressive retrieval, evidence alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pBK4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Associative Dreamer adds an associative memory to a recurrent world model. A reader retrieves information from earlier observations to help predict the consequences of an action, and an alignment loss teaches it which historical evidence to recover. The experiments use three visual tasks in which earlier events determine later outcomes.

Critical strengths
  • The history interventions are well designed: changing an earlier event while holding the current image and future actions fixed directly tests whether the model uses the relevant history, and disabling memory reads provides a complementary check.
  • On Transport and Power, evidence alignment substantially improves performance over the same memory architecture trained with prediction losses alone, showing that adding memory does not by itself yield useful retrieval. The paper is also clear that evaluation uses the training support and known retrieval times, while correspondence supervision is used only in the training loss and does not enter prediction.
Critical weaknesses
  • The experiments show that the memory path is used and that alignment improves over FW within the same fast-weight memory. They do not establish an advantage unique to the associative architecture over other memory designs. The paper acknowledges that this claim needs alternative-memory controls, but none are reported. A parameter-matched alternative—a compressed history buffer, for example, or attention over a learned summary—would help test whether the gain over the ordinary recurrent model reflects associative retrieval or access to more history.
Questions for authors
  1. Please compare the fast-weight memory with a parameter-matched alternative, such as a compressed history buffer or attention over a learned summary, trained with the same alignment loss.
Reviewer_fuWeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Associative Dreamer introduces a fixed-size associative memory into the Dreamer world model. It predicts action consequences by storing and retrieving historical visual information and optimizes memory retrieval by using an additional alignment loss. Experiments show that this method significantly outperforms the baseline model on some history-dependent tasks, but on the Assembly task, the improvement brought by the alignment loss is limited.

Critical strengths

1.The overall idea of the paper is relatively easy to follow. The idea of the problem to be solved is clear. In partially observable environments, due to previous events, the same current image and action may lead to different results. The motivating example (isolating the faulty power supply after the relevant clues are no longer visible) makes this problem concrete.

2.Rich in mathematical expression. The method is described very precisely, and the appendix derives it step by step in detail, providing a probabilistic basis for the auxiliary loss and enabling the method to be reproduced.

Critical weaknesses

1.The innovation points are not clear enough. There are already mature precedents for each component of the method (fast weight writing, multi-hop reading, annotation-supervised retrieval, EMA target).

  1. Its relevance in the real world is questionable. All the tasks are small and synthetic. This method is based on the retrieval timing and frame labels provided by the simulator, which are unavailable in real-world scenarios. No assessment was conducted on existing memory benchmarks or in real environments.

  2. The training and testing use the same trajectories and no data is held out, so it is impossible to distinguish between the model using the past and the model memorizing samples. In Appendix D.4, only some intermediate frames have been changed; the physical setting is the same as in training and thus cannot show generalization.

Questions for authors

Please ask the author to explain the core innovation of this article and supplement the experimental results based on unseen data and real-world scenarios.

Reviewer_SZ3QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper augments an RSSM with fixed-size fast-weight associative memory, an autoregressive Reader, and an auxiliary Value-alignment objective aligning retrieval means with EMA targets computed from annotated, query-relevant historical observations. Experiments on three controlled pixel environments show improved short-horizon open-loop consequence prediction over the ordinary RSSM on fixed training trajectories, with additional evaluation under local route perturbations in Transport and Power.

Critical strengths
  1. The controlled tasks directly test the use of historical information. Paired histories preserve the current observation and future actions while changing earlier relevant facts. Together with irrelevant-history interventions and future-read disablement, this provides meaningful evidence that the model uses historical information to predict consequences.
  2. The integration of memory and prediction is clearly specified. Alignment targets remain outside the prediction path, and the manuscript reports implementation details, seed variability, and unsuccessful runs. These details make the reported findings and their limitations easier to assess.
Critical weaknesses
  1. Generalization remains insufficiently evaluated. The main results measure performance on the training support. Although Appendix D.4 introduces unseen action prefixes, these perturbations preserve the evidence frames and subsequent outcomes. Consequently, the results provide limited evidence for generalization to unseen physical configurations, combinations of relevant facts, or longer memory delays.
  2. The contributions of the architecture and additional supervision are not sufficiently separated. EMA+V receives query-to-history correspondences unavailable to FW, but there is no RSSM or alternative-memory baseline receiving comparable supervision. Future-read disablement establishes reliance on the learned pathway, without establishing the specific benefits of matrix memory or autoregressive reading. On Assembly, EMA+V and FW differ by only 1.04 percentage points in mean balanced open-loop accuracy, which does not establish a reliable advantage from alignment in this task.
Questions for authors
  1. Does the advantage persist when training and evaluation are separated by physical configurations or combinations of relevant facts, rather than by local route perturbations that preserve the evidence?
  2. How does the method compare with an RSSM or a simple attention-based memory receiving the same historical-evidence supervision? Such a comparison would help isolate the contribution of the proposed memory architecture.
  3. In Assembly, what accounts for the gap between one-step prediction from an observed prefix and open-loop prediction? Beyond the aggregate Value-error diagnostics in the supplement, could a controlled analysis using the existing checkpoints distinguish errors in retrieved evidence from errors in how that evidence is used during the latent rollout?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.