acceptodds
Under review as a conference paper at ICLR 2027

Memory that Imagines: Learner Mental World Models as Memory for Long-Horizon Tutoring Agents

Abstract

Agent memory systems answer what happened. They store, retrieve and summarize interaction logs. A tutoring agent needs its memory to answer two other questions, what the learner knows now and what a candidate teaching action does. This paper casts the memory of a tutoring agent as a mental world model of the learner, a fixed-size belief state. The belief state supports three operations, WRITE (posterior update), READ (mastery with uncertainty) and IMAGINE (closed-loop rollout of candidate teaching sequences). Under a latent-state assumption, the belief retains all the evidence the history holds about future responses, whereas a log cut to any fixed budget can lose that evidence. The memory is a recurrent state-space model trained with closed-loop overshooting to roll forward on its own sampled responses. Two design choices adapt the model to learner data. Each exercise is bound to its response. Prediction and reconstruction use separate heads, because sharing one head costs 6.8 AUC points. On three knowledge-tracing benchmarks, this memory predicts the next 100 responses 8.5–16.2 AUC points better than a retrieval memory or a Bayesian Knowledge Tracing state, and better than two LLMs that read the full log. Over the same horizon, it matches knowledge-tracing models without a readable interface. In multi-session tutoring with simulated learners on ASSISTments 2017, planning with IMAGINE gains more on this memory than on a low-fidelity one in all ten protocol seeds. It lands above random sequencing on the former and below it on the latter. An LLM tutor reading the serialized READ interface also beats random sequencing, with both providers. Memory size is O(1) in history length, versus O(t) for logs.

Primary area:
learning on time series and dynamical systems
Keywords:
agent memory, learner modeling, world models, knowledge tracing, intelligent tutoring systems, planning, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1LNHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes that a tutoring agent’s memory for personalized education should be a mental world model of the learner. The world model is a fixed-size belief state that supports three operations, (a) write to update the belief after new exercise and response; (b) read to retrieve per-concept mastery; (c) imagine to roll a candidate exercise sequence forward and predict learner’s responses and mastery gain. Four experiments, on three common knowledge-tracing datasets, evaluate the memory fidelity, tutoring value, and memory cost.

Critical strengths
  1. The presentation and figures are very clear and easy to follow. I especially think fig.1-2 help me a lot in understanding the pipeline.
  2. The experiment results show clear evidence that world-model memory keeps both item-specific and learner-speciic information, while other readable memories lose one of them. I think this is very important for tutoring, since a tutor needs to know both which learner is behind and which exercises they will fail.
Critical weaknesses
  1. Based on the results, it seems like that world-model memory is not clearly different from, or better than, a knowledge-tracing model. The memory is a recurrent model so it is close to all deep-learning-based knowledge tracng models after DKT. The paper argues that the novelty lies in that the memory can be read and rolled forward. But appendix shows that these can also be added to DKT directly. I assume all KT models with a hidden state (which summarizes the history information) and an approxiamately learned forward dynamics (here is the recurrent model) should be able to do this?
  2. Related to my point 1, I want to understand the operation “imagine” better. I know this is a quite popular way in (deep) RL where you do the roll out to learn or select a policy. Here, “imagine” predicts the mastery gain by rolling the memory forward on its own sampled responses, but a change in mastery can mean either that the learner learned, or only that the memory updated its belief about what the learner already knew. Is there anything in the model that separates the two? The memory is also trained on logged sequences whose exercise order was not randomized, yet the planner uses it to rank candidate sequences that may not appear in the data. How do the authors know that the predicted mastery gain ranks candidates by learning gain? For example, does it predict the realized probe gain across the planner’s 32 candidates?
  3. All learners are simulated, and the learning gain is the change in correctness predicted by an AKT model. I am worried that the evaluation setup is too weak to support the tutoring results.
Questions for authors

See weakness.

Reviewer_qmuoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about memory for LLM agents that act as personalized tutors. The agent’s job is to pick which exercise a student should practice next. Standard agent memory stores past interactions and retrieves them later. The paper argues this is the wrong kind of memory for a tutor, because choosing the next exercise depends on two things the history does not state directly, which are 1. what the student currently knows and 2. how the student would respond to a candidate exercise.

The proposed approach replaces the interaction log with a learned model of the student and uses that model as the memory. Specifically, the memory is a fixed-size belief state with three operations. WRITE updates the state after each exercise and response. READ outputs an estimate of the student’s mastery on each concept, with uncertainty. IMAGINE takes a candidate sequence of exercises and predicts how the student would respond and how much mastery would improve. The model is a recurrent state-space model in the style of Dreamer. It is trained with a closed-loop overshooting objective so that multi-step predictions stay accurate when the model feeds its own sampled responses back in, which is how it runs at planning time. Three propositions argue that a belief state preserves information a token-budgeted log can lose, and that planning with an inaccurate model can do worse than random selection.

Experiments use three knowledge-tracing benchmarks (ASSISTments 2009, ASSISTments 2017, NIPS34). On predicting the next 100 responses, the memory outperforms a retrieval baseline and Bayesian Knowledge Tracing by 8.5 to 16.2 AUC points and is on par with DKT and AKT. In a simulated multi-session tutoring protocol, with an AKT model acting as the student, a random-shooting planner using IMAGINE slightly outperforms random exercise selection on the proposed memory and underperforms it on the BKT memory.

Critical strengths
  1. Memory you can read and roll forward is what an agent needs when it makes decisions about a person over time, and the shared three-operation interface lets the same policy run on every memory, so differences can be attributed to the memory.
  2. The experiments are thorough and clear.
Critical weaknesses
  1. I don't see the connection between the propositions and the paper's actual method. Specifically, if I understand correctly, Proposition 1 is the Markov property plus data processing, Proposition 2 is a hand-built counterexample against one specific retrieval rule, and Proposition 3 is a hand-built counterexample too. None of them says anything about a learned belief state, which is what the paper actually uses.
  2. The proposed memory has 688k parameters against 526k for DKT and 434k for AKT. I think it would make the comparison more apple-to-apple if the sizes are the same.
Questions for authors
  1. Why does showing imagined outcomes hurt the LLM tutor in all four conditions? If only the random-shooting planner benefits from IMAGINE, is it fair to call it a general memory operation?
Reviewer_mVKhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper casts the memory of a tutoring agent as a mental world model of the learner, a fixed-size belief state, which supports WRTIE, READ, and IMAGINE operations. The memory is a recurrent state-space model trained with closed-loop overshooting to roll forward on its own sampled responses. The proposed model shows improvements over baselines on multiple knowledge-tracing benchmarks.

Critical strengths
  1. The WRITE, READ, and IMAGINE abstraction is intuitive and easy to understand. The paper also makes it clear that all compared memories expose the same interface.
  2. The paper reports strong results on three knowledge-tracing benchmarks.
  3. The paper also reports additional details and ablations to analyze the components of the proposed framework.
Critical weaknesses
  1. The learned recurrent state, per-concept mastery readout, next-response predictor, and simulated rollout are all closely aligned with standard knowledge tracing and model-based sequencing. Calling the state “memory” and exposing it through three APIs is a sensible architecture, but the paper does not convincingly isolate what is technically novel. Much of the claimed contribution appears to be a general interface around existing learner models rather than a distinctive new model.
  2. The key world-model planner gain over random sequencing is approximately +0.013 on ASSISTments 2017. On ASSISTments 2009 and NIPS34, the planner’s advantages are within noise. How do these results affect the general conclusions proposed in this paper?
  3. Although the paper makes commendable efforts to reduce direct self-evaluation, all tutoring claims remain within a family of knowledge-tracing models trained on the same interaction environment and binary correctness observations. How can these results support general conclusions made by the paper?
  4. The theoretical results provide limited support for the proposed method. While the propositions establish the sufficiency of exact Bayesian beliefs and potential failures of retrieval memory and response prediction, they do not show that the learned state approximates a sufficient belief or that the evaluated baselines exhibit these failures. Thus, the theory motivates belief states in principle but does not directly validate the proposed method or its downstream claims.
Questions for authors

Please see weaknesses for my questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.