acceptodds
Under review as a conference paper at ICLR 2027

Beyond Present Recall: Auditing Future Plastic Response under Reconstructive Writeback

Abstract

When neural memory architectures feed reconstructed content into plastic writeback, using stored memories alters the substrate for later queries. Within one controlled recurrent executor, matched interventions establish that prespecified present observables and writeback trajectories do not uniquely determine future response. Structural routing and runtime retrieval state causally reshape memory-wise response: the runtime contrast persists across identical stored traces, ruling out different written histories as the explanation in this executor. Common-state evaluation provides an expected attribution consistency check. Supporting stages show that reconstructive writeback selectively redistributes future memory quality, resolving excess memory-wise dispersion while aggregate excess remains unresolved at the primary comparison. At a fixed bank and schedule, a frozen predictor forecasts held-out quality changes across independent error directions, with post-freeze decomposition attributing this association to its direction-averaged component while stream-specific residuals do not transfer. In a separate differentiable system, post-hoc individual-memory calibration tests under oracle-residual inputs show known trace decay accounts for most calibration gain over an unweighted local baseline. Externally, visual memory replicates response redistribution and frozen prediction, whereas native same-checkpoint focal-layer test-time training preserves centered ordering and sign despite absolute common-mode failure. These findings motivate auditing memory-wise changes under specified writeback and retrieval conditions, alongside present recall and aggregate scores.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
writable neural memory, reconstructive writeback, learning dynamics, memory plasticity, test-time learning, continual adaptation, mechanistic analysis, prospective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.60 over 5 reviews, above 10% of the others.
Reviewer_muytOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to address how reconstructive writeback changes the future quality of its individual memories. It uses response operators to predict these changes and controlled interventions to examine the effects of structural routing and retrieval conditions. Experiments are performed over synthetic memory systems, visual writable memory, and a focal layer of a pretrained test-time training (TTT) model.

Critical strengths
  1. The paper tries to address the limitation of evaluating writable memory solely via present recall and aggregate scores.

  2. The matched experiments isolate the effect of retrieval conditions by keeping the written updates and stored traces identical. This helps distinguish changes in stored information from changes in how that information is retrieved.

  3. The authors clearly report negative results and limitations, including cases where simpler baselines explain the findings or the proposed predictions fail. This makes the conclusions easier to assess.

Critical weaknesses
  1. The main experiment shows that changing the retrieval operator changes future responses despite matching selected present measurements. However, the paper needs to explain more clearly what general insight about memory dynamics follows from this result.

  2. The prediction experiment tests whether a response estimated using one set of reconstruction errors can predict the effects of another set. However, removing the part of the predictor that depends on error direction barely changes performance. This suggests that the strong results mainly reflect effects shared across errors. Since the stored memories and update order remain fixed within each comparison, it is unclear whether the predictor works when these conditions change.

  3. Full propagation is used to track how each update affects later memory states. However, a simpler predictor that accounts for how much earlier updates decay over time performs almost as well. This baseline is missing from the later tests of changes across memories and unseen error directions, so those results do not establish the additional benefit of full propagation.

  4. The TTT experiment tests whether the predictions remain useful in a pretrained model. Here, the predictor fails to estimate the actual changes in memory scores. After the average change is removed, it can partly identify which items fare better or worse than others, but the size of these differences remains unreliable. The result therefore supports relative comparisons more than reliable prediction of future scores.

  5. The experiments do not establish whether the audit can identify meaningful memory failures. The main causal result comes from one constructed system, visual-memory accuracy remains perfect, and the TTT evaluation measures reconstruction at a single layer. The paper therefore does not establish whether the reported score changes help identify actual recall failures or poorer performance on the model’s main task.

  6. The writing in the paper is difficult to follow. I understand Generative AI tools were used to refine the paper but I feel it hampers the readability and comprehensibility of the paper. The content switches between memory systems with different update rules and quality scores, making it hard to connect the experiments to the main claim. A simpler problem statement, clearer descriptions of the experimental setups, and a direct explanation of what each result establishes would improve readability.

Questions for authors
  1. What insight about memory dynamics can be inferred from the controlled retrieval experiment, and what evidence suggests it applies beyond the constructed system?

  2. Does the predictor remain reliable when the stored memories or update order change, given that the error-direction component contributes little to its performance?

  3. How does full propagation compare with the decay-weighted baseline in the tests of changes across memories and unseen error directions?

  4. Why does the native TTT predictor fail to estimate actual score changes despite retaining some ability to rank the changes across items?

  5. Do the predicted score changes correspond to actual recall errors or poorer performance on the model’s main task?

  6. Could you provide a concise overview linking each experiment’s memory system, update rule, and quality measure to the claim it supports?

1 comment

Thank you to the authors for the detailed response and reporting additional experiments. My assessment is as follows -

  1. The new experiment explains how changing the retrieval process can change future recall even when the stored memory is identical. This helps clarify the mechanism. However, the result has only been demonstrated in the constructed system, and its relevance to trained memory models remains unclear.
  2. The prediction results use a fixed set of memories and a fixed update order. The rebuttal confirms that prediction accuracy when either changes has not been tested. This question therefore remains open.
  3. The new comparison shows that full propagation reduces prediction error by 14.8% relative to the decay-weighted baseline on the oracle-residual test. It also improves prediction for random error directions. This addresses my request for the missing comparison.
  4. Adding the quadratic term corrects whether the average memory score increases or decreases in all 12 TTT windows. This helps explain why the original predictor failed. However, the sizes of the predicted changes remain inaccurate.
  5. The synthetic stress test produces actual recall errors under large perturbations, but the local predictor fails to forecast those errors. In the pretrained TTT model, the new result shows that the predictor can identify tokens whose reconstruction error worsens at one layer. Whether it can predict recall failures or increased language-model loss remains untested.

The additional experiments address several specific questions. My main concerns about prediction under changing memory conditions and the connection to actual task failures remain. I therefore maintain my original rating of 1.

Reviewer_LWq2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how reconstructive writeback changes the future quality of individual memories, beyond what present recall or aggregate scores reveal. It uses local response analysis to characterize these changes and constructs matched interventions showing that structural routing and retrieval state can alter future responses despite matching specified present observables and written traces. Synthetic experiments support selective redistribution and conditional response prediction, with additional visual memory and pretrained TTT experiments establishing more limited external replication.

Critical strengths

The study examines several aspects of reconstructive writeback, including changes to individual memories, response prediction, and the effects of structural routing and retrieval state. It also reports useful controls and negative results.

Critical weaknesses
  1. The experiments do not clearly establish a substantial advance in understanding reconstructive writeback. Section 4 finds that prediction is largely explained by a component independent of error direction or known trace decay, but does not sufficiently explain what these results reveal about memory dynamics. Section 5 demonstrates response differences in a deliberately constructed system, while Section 6 does not establish their relevance to consequential memory behavior. Although these limitations are acknowledged, the paper needs a clearer synthesis of what new insight the experiments collectively provide.
  2. The presentation often reads as a sequence of experimental reports rather than a connected scientific argument, particularly in Section 4. Numerical metrics, protocol details, and qualifications receive more emphasis than the motivation, interpretation, and implications of each experiment. Changes in systems, reference trajectories, and quality measures are also difficult to track. The abstract similarly compresses many results and qualifications without clearly introducing the central problem and contribution. As a result, it is hard to identify which hypothesis each experiment tests and how the findings support the central contribution.
Questions for authors

Could the authors clarify the central research question and how Sections 3–6 jointly address it? In particular, what role does each section play in the overall argument, and what main conclusion emerges from their combined findings?

Reviewer_38sfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how each stored memory changes in the future when reconstructed content is written back into a plastic memory. It measures signed per-memory change ΔQi\Delta Q_i against a matched reference and studies a local response operator VV and its centered form GG. Small synthetic recurrent and fast-weight memories show more across-memory dispersion with reconstruction error while mean support does not move, and a frozen susceptibility predicts per-memory change under resampled error directions. In a 12-dimensional executor, changing off-manifold routing or runtime state changes VV while declared present observables and writeback are matched. A CIFAR-10 slot memory and the layer-6 TTT-MLP-125M executor give partial replication.

Critical strengths

The experimental discipline is unusually careful. There are clean-gate checks, frozen predictors, seed-cluster intervals, and failed or stopped tests are kept and reported (Appendix H, e.g. the B-ORTHO clean-scale confound). The paper is honest about scope, for example the TTT absolute prediction failure and the post-hoc decay control that explains most of the calibration gain in Section 4.2. Separating local and propagated response (slope 0.491 vs 1.007) is a clear and useful result.

Critical weaknesses

Key tests are weaker than they look. In B-OOS the write split in Appendix C.1 has a direction-free part (ϵ/32)rt0(h~t0)⊤(\epsilon/32) r_t^0 (\tilde h_t^0)^\top and an error part that is only order ϵ2\epsilon^2. So the held-out direction test mostly checks that writing the clean reconstruction with gate strength ϵ\epsilon is predictable, and the paper itself finds the directional term explains at most 9.51×10−69.51 \times 10^{-6} of predictor variance. In the gated Hebbian replication the clean arm writes nothing (Ht=0H_t = 0), so "excess dispersion from error" is mixed with "writing vs not writing". The causal result in Section 5 is close to true by design. Changing an off-manifold routing block of a 12-dimensional tanh network will change its Jacobian even when a fixed set of present observables match, and the washout check is exact zero by determinism, as the authors note.

The evidence on real models is thin. Visual memory is one operating point with future accuracy 1.0, and on TTT the centered no-fit R2R^2 is 0.060 with window-cluster CI [-0.200, 0.274], while the uncentered prediction gets the common shift sign wrong (+0.031 predicted vs -0.014 observed). Some results come from reference reconstructions where the original raw tables are missing (Table 4). Related work misses retrieval-induced forgetting (Norman et al., 2007), reconsolidation in associative memory (Nowicki et al., 2013), per-example forgetting (Toneva et al., 2019) and self-consuming training loops (Shumailov et al., 2024; Alemohammad et al., 2024), which study very similar effects of reusing recalled or generated content.

Questions for authors
  1. In B-OOS, how well does a predictor built only from the direction-free clean write (no error direction at all) do? If it matches, the test is about writeback strength, not error content.
  2. In the gated Hebbian setting, can you add a control arm with the same gate magnitude but zero error direction, to separate error from write amount?
  3. Is there a trained model case (two TTT or Titans configurations, or two checkpoints) with matched present recall but clearly different future response? This would show the Section 5 result matters beyond a hand-built executor.
  4. What causes the TTT common-mode sign error, and does the centered result hold on other layers or a second checkpoint?
Reviewer_83h7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about how reconstructive writeback affects the future quality of individual memories in neural systems with plastic internal states. It shows that writeback can redistribute memory quality without producing a statistically resolved change in aggregate performance. Using sensitivity-based response operators and controlled interventions, it examines how structural routing and retrieval-time state influence these changes. Experiments across recurrent memory systems, visual memory, and a pretrained test-time-training model assess the predictability and generality of these effects.

Critical strengths
  • Careful isolation of reconstructive writeback effects
  • individual-memory behavior
  • treatment of limitations and negative findings
Critical weaknesses
  • The response operator V and G=V^\top PV may be mathematically appropriate, but their properties follow from established sensitivity analysis. The paper’s theoretical contribution appears to be the application of these concepts to reconstructive writeback rather than the development of a fundamentally new method.

  • The strong predictive performance may largely reflect a stable memory-wise response pattern rather than sensitivity to the particular perturbation being applied. In Section 4.1 the predictive correlations are Pearson 0.974, Spearman 0.997 and Sign Acc.99.4%. However, the post-freeze decomposition shows that the direction-independent component S_i^{(0)} achieves essentially identical performance. The direction-specific residual has Pearson correlation approximately -0.138. My interpretation is that the result establishes a memory-specific susceptibility component which transfers across independently sampled error directions under a fixed bank and schedule. It does not demonstrate successful prediction of the consequences of different error directions.

  • The relationship between the proposed response operator and existing research on future-response sensitivity in stateful learning systems needs further discussion. Prior approaches have examined how perturbations to hidden learning states propagate through subsequent optimization steps and affect future behavior. Clarifying the methodological differences and specific contributions of the present formulation would help establish its novelty.

  • The finite-horizon calibration improvement seems to be explained by known decay (Section 4.2), which in turn blurs the interpretation that modeling complex recurrent interactions is essential to the observed predictive performance

  • The work reports prediction under controlled exogenous perturbations and known dynamics but the writeback experiment on the pretrained TTT model presents a substantially weaker result.

  • Also despite the demonstration in changes in continuous memory-support scores, these changes are not consistently associated with task-level failures or improvements. Eg, the visual-memory experiment reports positive redistribution while future accuracy remains 1.0. This raises a practical question: When does the proposed redistribution metric identify consequential changes in memory performance and no small variations in a continuous diagnostic?

Questions for authors

Q1 (a) What is the main methodological contribution beyond established first order sensitivity analysis and known results on plastic-state-dependent future behavior? (b) Which findings are specific to reconstructive writeback and not general consequences of stateful learning dynamics?

Q2 Given that cross direction prediction is almost entirely explained by the direction-independent susceptibility component, what additional information does the proposed predictor provide about the effects of particular reconstruction errors?

Q3 (a) Can the structural routing and retrieval state effects be demonstrated in the pretrained TTT system or another practical memory architecture? (b) can the proposed response metrics predict consequential memory degradation or task level failures rather than only continuous support changes?

Q4 The response operator is computed using complete knowledge of the underlying plastic dynamics and reference trajectory. (a) How sensitive are its predictive accuracy and calibration to incomplete knowledge of the internal learning state or update dynamics? (b) Is it possible to identify a minimal set of observable quantities sufficient for predicting future memory wise responses, and (c) how does the prediction error evolve as information about the hidden state becomes unavailable?

Reviewer_zQRDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how reconstructive writeback changes individual stored memories beyond present recall and aggregate scores. It combines local response analysis with controlled interventions that vary structural routing or retrieval conditions while matching declared present observables and writeback traces. Synthetic, visual-memory and focal-layer TTT experiments test redistribution and conditional prediction, with explicit boundaries: direction-averaged effects and trace decay explain much of the predictive success, while native TTT retains centered ordering but fails absolute prediction.

Critical strengths
  1. The controlled experiments distinguish changes in stored traces from changes in their expression. Matching every plastic increment and the final trace makes the runtime contrast informative, while the common-state check verifies the intended attribution (§5). The paper also carefully limits present matching to declared observables rather than claiming complete behavioral equivalence.

  2. The evaluation separates memory-wise dispersion from aggregate change and reports uncertainty at the seed or window level. The simpler predictive controls, failed interaction hypothesis, and TTT common-mode failure are openly reported (§§3–6). This makes the supported diagnostic findings easier to assess and provides useful evidence about where first-order response analysis succeeds or breaks down.

Critical weaknesses
  1. The conceptual and predictive advance remains limited. The matched interventions in §5 provide a useful existence demonstration, but changing the off-memory-manifold retrieval block does not yet explain when hidden response differences become consequential. Moreover, the direction-averaged predictor matches the full susceptibility (§4.1), and known decay accounts for nearly all individual-memory calibration gains (§4.2). These findings leave unclear what additional insight the detailed response analysis provides beyond the simpler explanations. A sharper mechanistic conclusion, supported by matched comparisons on the remaining prediction tests, would strengthen the contribution.

  2. The external evidence establishes proxy-score changes more convincingly than task-relevant memory degradation. Visual retrieval remains perfectly accurate (§6.1), while TTT tests one layer of one checkpoint on 12 windows using a reconstruction score (§6.2). Under native updates, absolute prediction fails and centered magnitudes remain uncalibrated. Relative ordering is informative, but it is unclear whether this audit identifies consequential failures better than simple present-state diagnostics. A focused task-level check would strengthen its significance.

Questions for authors
  1. Which result in §5 establishes more than sensitivity to a changed off-manifold retrieval map? Can you identify a condition, such as writeback magnitude or retrieval horizon, under which the matched response difference becomes consequential or vanishes?

  2. Does the decay-weighted local baseline also match full propagation on the finite-error population and oracle-residual tests in §4.2? A paired comparison using the same inputs and outcomes would clarify what state-dependent propagation adds beyond known decay.

  3. Does ranking memories or tokens by predicted deterioration track downstream recall or loss changes better than ranking them by current support or reconstruction error? A focused comparison in one existing external setting would help establish the practical meaning of the audit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.