acceptodds
Under review as a conference paper at ICLR 2027

Revisiting What, Where, and When in Latent Memory Generation for Frozen LLM Reasoners

Abstract

Latent memory generation offers a promising approach to augmenting the test- time reasoning of frozen large language models, in which an auxiliary Weaver synthesizes continuous memory representations and a Trigger selectively injects them into the reasoning stream. However, existing implementations rely on three convenient but weakly justified defaults: the Weaver is trained with brief ratio- nales, latent memory is read from the final Transformer layer, and memory recall is considered only at punctuation boundaries. We revisit these choices through a unified What–Where–When framework. For what to remember, we propose CoT- Supervised Memory Compression (CSMC). CSMC first trains the Weaver with richer CoT supervision and then introduces task-driven compression during Trig- ger reinforcement learning: CoT supervision improves 15 of 16 evaluated con- figurations, while compression further improves eight using fewer memory vec- tors. For where to extract, we propose Depth-Aware Readout (DAR) and show that intermediate-layer readout outperforms final-layer readout in 11 configura- tions and matches it in one, with a macro-average gain of 4.50 percentage points. For when to recall, we introduce the Reasoning-State-Aware Trigger (RSAT), which replaces punctuation-based heuristics with predictive-entropy-based acti- vation and improves StrategyQA accuracy from 63.17% to 76.13% on Qwen2.5- 7B-Instruct. Together, these results recast latent memory generation not as a fixed architecture but as an optimizable design space spanning what information is dis- tilled and retained, where memory is read out, and when it is recalled, providing a more systematic foundation for augmenting the reasoning of frozen language models. The code will be released at https://github.com/anonymous-researcher- coder/latent-memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Latent Memory Generation, Frozen LLM Reasoning, Instructional Chain-of-Thought, Depth-Aware Readout, Reasoning-State-Aware Trigger, Predictive Entropy, Test-Time Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NND4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits latent memory generation through the What–Where–When perspective. On four Qwen architectures and four inference benchmarks, the authors verify that their method performs better under most evaluation configurations.

Critical strengths
  1. Relatively sound writing and presentation.
  2. From the results shown in this paper, the effect of the method is considerable.
Critical weaknesses
  1. The motivation of this paper needs further clarification. In general, research focuses on a class of strategies and methods that can solve the general problem in this field. However, the method proposed in this paper seems to be entirely incremental optimization for MemGen, and the proposed methods are also designed for the components in MemGen, which makes it difficult to agree with the rationality of the motivation.
  2. Furthermore, the authors analyze issues from a specific approach and scenario and solve them, which is more like an optimization of MemGen rather than new research work.
  3. It can also be seen from Section 3 that the innovation of the paper is designed for each module of MemGen. In this case, the paper does not introduce any comparison method (in other words, there is no other comparison method for this specific case), so the author needs to further clarify the motivation and research value of this paper. Or introduce more comparative methods to highlight technical contributions.
  4. Although providing code is not required, the paper provides links to a broken repository.
  5. The chart in the paper should include more captions, such as experimental setup, background, and a brief conclusion, to improve readability.
Questions for authors

My main questions concern the article's motivation and actual contribution. For me, this is an incremental optimization of MemGen, not an innovative work. The author needs to answer the following questions:

  1. Can the proposed approach be adapted to other frameworks?
  2. How can technical contributions be highlighted in relatively niche research scenarios?
  3. Is the proposed method optimized and designed for MemGen, or is it designed and optimized for problems that are common to the field?
  4. Is the proposed approach also applicable to future frameworks?

In general, the overall quality of the paper is considerable, but the author needs to clarify my concerns further and provide data support, and I will consider raising the score.

8 comments
AuthorsOpenReview ↗

Response to Reviewer NND4 [Part 1/3]

We thank the reviewer for the constructive feedback on the motivation, contribution, and broader applicability of our work. We respond to each concern below.

Response to W1:

We agree that the current presentation does not sufficiently distinguish the broader research questions from their implementation in MemGen. Our motivation is to understand how generated latent representations can effectively support a frozen reasoner. This involves three key choices: what the memory generator learns and retains information, where the generated memory is represented and passed to the reasoner, and when the memory should be supplied during reasoning.

We chose MemGen because its architecture exposes all three choices within one system, enabling controlled comparisons while retaining the same frozen reasoner and memory generation interface. We will revise the Introduction to establish these research questions before introducing MemGen as the experimental platform, and clarify which conclusions have been demonstrated within MemGen and which require further validation across frameworks.

Response to W2:

The contribution lies in examining how supervision, representation extraction, and memory access affect the usefulness of latent memory under a frozen-reasoner constraint.

For example, our CSMC compression can also be understood as a process of denoising memory. By aiming to reduce redundancy and noise in memory representations, CSMC provides a potentially useful perspective for studies on latent memory. More broadly, existing memory-related work studies how to construct memories, how to retrieve them, and when to access them. Our RSAT investigates this timing question in latent memory generation. Specifically, token-level entropy can be used to characterize the model's reasoning state and identify candidate positions for consulting the memory controller. This criterion could be extended to a wide range of memory-based methods, providing a general mechanism for adaptive memory access.

Together, these comparisons address distinct design questions rather than attributing all improvements to a single modified system.

Response to W3:

We agree that comparisons restricted to MemGen variants are insufficient to establish broader applicability. We are supplementing the evaluation with a frozen-reasoner baseline using plain CoT prompting, the combined CSMC–DAR–RSAT system, and component ablations. These experiments cover Qwen2.5-7B on GPQA with and SVAMP(86.9% is the Qwen2.5-7B-Instruct result reported in CARFT (Zhu et al., 2025)) three random seeds and a maximum generation length of 1,024 tokens.

Table R1. Supplementary accuracy (%) under matched evaluation settings.

BackboneDatasetNo memoryMemGenCSMC + DAR + RSATCSMC + DARCSMC + RSATDAR + RSATCSMCDARRSAT
Qwen2.5-7B-InstructGPQA25.25 ± 0.0025.59 ± 2.2823.91 ± 1.2728.11 ± 2.7827.78 ± 0.8722.56 ± 1.1727.78 ± 1.5224.07 ± 0.7725.76 ± 2.20
Qwen2.5-7B-InstructSVAMP86.982.33 ± 6.3588.44 ± 3.2988.78 ± 2.3687.78 ± 3.2986.67 ± 7.5486.67 ± 2.3384.22 ± 5.4084.78 ± 3.67

In future work, we will compare our method with other relevant approaches. However, due to current resource constraints, we first conducted the relevant ablation experiments to analyze the contribution of each component.

Response to W4:

We apologize for the difficulty accessing the repository. The URL in the submitted PDF was split across lines without an embedded hyperlink, which may have caused some PDF readers to recognize an incomplete address. We will correct this in the revised manuscript. The correct link is https://github.com/anonymous-researcher-coder/latent-memory.

We are still conducting ablation experiments and preparing the code for release. We plan to organize and upload the code, with the release tentatively scheduled for December 2026.

Response to W5:

Thank you for this suggestion. We will expand the captions to specify the experimental setting, comparison being made, evaluation metric, and principal observation.

For example, the caption of Table 1 will explicitly distinguish the effect of CoT supervision from the additional effect of compression and state the corresponding memory budget reductions. The captions of Tables 2 and 3 will specify the readout depth and triggering comparisons, respectively. The supplementary tables will also identify the number of seeds and explain how variability is reported.

AuthorsOpenReview ↗

Response to Reviewer NND4 [Part 2/3]

Response to Q1:

DAR provides a concrete example of a component that could be adapted to another framework. SoftCoT obtains continuous representations from an auxiliary model and maps them into a frozen reasoner through a trainable projection. This provides an interface in which the layer used to extract these representations can be varied. SoftCoT

A simple transfer experiment would compare final and intermediate layer readout in SoftCoT under matched training and evaluation conditions, with the projection trained separately for each configuration. This would test whether our findings on readout depth extend beyond MemGen. Such an evaluation is beyond the scope of the current study, and we leave it as a direction for future work.

Response to Q2:

Although our experiments use MemGen, the practical need they address is broader: how to supply useful information to a frozen reasoner through compact latent representations. This need arises in several application settings.

  • Reasoning augmentation: Auxiliary models can provide continuous representations that support the reasoning of a frozen model, as illustrated by SoftCoT. Our findings can inform decisions about the reasoning information used for supervision and the representations supplied to the reasoner, both of which matter when enhancing reasoning without updating the main model’s parameters.
  • Retrieval augmented generation: Frameworks such as CARE encode retrieved context into compact memory representations. In this setting, the amount and relevance of the supplied information are important design concerns. Our emphasis on selective retention addresses a related need: compression guided by task feedback aims to preserve useful information while reducing redundancy and the influence of irrelevant content. CARE
  • Extended multimodal reasoning: As generation unfolds, a model may need to reuse visual evidence or reasoning constraints. DLMR illustrates this need through dynamic latent memory injection. Our investigation of invocation timing provides a design direction for determining when such information should be made available during reasoning. DLMR

These examples illustrate shared application needs and identify systems that could potentially benefit from our findings. We will make these connections more explicit in the revised manuscript, while distinguishing potential applications from the settings empirically evaluated in this work.

AuthorsOpenReview ↗

Response to Reviewer NND4 [Part 3/3]

Response to Q3:

Our methods are motivated by design problems shared by a class of latent memory augmentation systems, and MemGen provides the platform on which we implement and evaluate them. The underlying questions concern how to preserve useful reasoning information, select suitable memory representations, and determine when memory support should be provided.

ComponentShared research problemHow we investigate it in MemGen
CSMCHow can useful reasoning information be learned and retained under a limited memory budget?We examine richer CoT supervision and separately study the effects of compressing the generated memory before injection.
DARWhich internal representations are suitable for supporting downstream reasoning?We compare intermediate and final layer readout while retaining the same supervision and memory budget.
RSATHow should memory access respond to the evolving needs of a reasoning process?We use predictive entropy to select opportunities for consulting the memory controller, extending consideration beyond punctuation boundaries.

For example, punctuation gating is a specific choice in MemGen. We choose predictive entropy because it reflects the model’s current predictive uncertainty at each decoding step. Beyond its role in RSAT, this choice highlights a broader research perspective: memory access can be studied in relation to the model’s evolving state and potential need for additional support. This perspective may inform other work on memory augmentation by encouraging the joint consideration of memory quality and the timing of its use, alongside improvements in memory construction and retrieval. Predictive entropy provides one concrete signal for exploring this relationship. Similarly, the supervision and readout choices in MemGen instantiate broader questions about memory content and representation.

We chose MemGen because it exposes all three choices within one architecture and allows controlled comparisons with a frozen reasoner. Our implementations and current empirical evidence are based on this platform, while the research motivation concerns these broader design problems.

Response to Q4:

We expect these design principles to remain relevant to future frameworks that use latent memory to support a frozen reasoner. They can guide the design of memory interfaces as information sources, memory generators, and invocation controllers become more diverse.

  • What to remember: Future frameworks may combine information from retrieved documents, interaction histories, and specialized memory generators. These sources can contain overlapping information or content that is irrelevant to the current task. Applying this principle would involve selecting and consolidating useful information within the available memory budget, while reducing redundancy and noise. The selection criteria could also adapt as the task or available information changes.
  • Where to extract: A framework may employ memory generators with different architectures, depths, or modalities. For generators that expose representations at multiple depths, the readout choice could be adapted separately for each generator according to how effectively its representations support the downstream reasoner. A shared projection interface could then map these different representations into the reasoner’s input space. This would allow a framework to accommodate heterogeneous generators without requiring them to use the same extraction depth.
  • When to recall: Future controllers may jointly decide whether to invoke memory, which source to access, and how much information to inject. The reasoner’s uncertainty could serve as one input to this decision, alongside the available memory sources and computational budget. For example, an uncertain reasoning step could prompt the controller to consider relevant memory, while the controller determines whether an invocation is useful and which source is most appropriate.

In these settings, adaptation would involve incorporating the design principles into the new framework’s memory and control interfaces. The specific readout depths, compression ratios, and uncertainty thresholds would be selected and validated for the target models and tasks. This provides concrete directions for future evaluation as memory architectures evolve.

Reviewer_oqT2OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper builds on MemGen, where a frozen reasoner is helped by a Weaver that produces latent memory tokens and a Trigger that decides when to insert them. It revisits three default choices. For "what", it trains the Weaver on chain-of-thought generated by GPT-4o instead of the short dataset rationales, and adds a learned compressor that merges groups of memory vectors (CSMC). For "where", it reads the memory from the middle layer of the Weaver instead of the last layer (DAR). For "when", it lets the Trigger fire where the next-token entropy falls in a percentile band, instead of only at punctuation (RSAT). Each change is tested on its own with four Qwen models and four datasets.

Critical strengths

The paper asks reasonable questions about a recent framework, and each change is compared to a matched baseline over 16 model and dataset pairs. The middle-layer readout result is the most interesting part. It is a very simple change that helps in 11 of 16 cases, sometimes by more than 10 points, but these are single runs, so it is hard to know how much of this would hold

Critical weaknesses

I have concerns about the baseline numbers and the evaluation. There is no result for the frozen reasoner without any latent memory, so we cannot tell whether MemGen or the proposed changes help the base model at all.

The Qwen2.5 technical report, (Yang et al., 2024a), gives 91.6% on GSM8K for Qwen2.5-7B-Instruct, much higher than the 73.77% baseline here, although its evaluation setup is different. This makes me think part of the gains may come from fixing answer format or extraction problems, especially for the CoT supervision.

Also, all results are from a single run. On 198 GPQA questions a 1 to 2 point gap is only 2 to 4 answers.

Each change is a small modification of MemGen, so I see the novelty as limited, and the CoT supervision gain is mostly distillation from GPT-4o. Compression does not help in half of the cases and drops accuracy by up to about 9 points. Only one intermediate layer is tested.

RSAT has one main setting, the entropy band, and the paper tries four values of it. The default band [P80, P95) is the one that ranks best on the same results that are reported in Table 3. Therefore, the band is chosen by looking at the final results, so the reported average gain of 2.05 points may be higher than what a new user would get.

The three parts are also never combined into one system. Each one is tested alone, with the other two kept at the MemGen defaults, and the Discussion leaves joint adaptation for future work. Because of this, to me the paper reads more like an ablation study of MemGen's design choices than an evaluation of a new method. We don't know whether CSMC, DAR and RSAT add up, cancel out, or hurt each other when used together, and there is no final system to compare with MemGen.

Without a no-memory baseline, and with only a single run per setting, I can't tell whether the reported gains are real improvements, and this is the main reason for my rating.

Questions for authors

Can you show results with all three changes used together, with the RSAT band chosen on a held-out validation split?

What is the accuracy of each frozen reasoner with plain CoT prompting and no latent memory?

How are answers that cannot be parsed counted? Which GPQA data was used for training, and does it overlap with the 198 test questions?

Could you report the mean and standard deviation over a few seeds for the main tables?

7 comments
AuthorsOpenReview ↗

Response to Reviewer oqT2 [Part 1/3]

We thank the reviewer for the careful review and constructive suggestions. We respond to each weakness and question below. Before addressing the individual comments, we summarize the updates to our evaluation and the supplementary experiments.

We acknowledge the reviewers’ concern that issues with answer formatting or extraction may have affected the original evaluation. In the supplementary experiments, we use revised instructions for formatting answers and apply the same extraction and scoring rules to all methods in our own reruns.

The supplementary evaluation covers Qwen2.5-7B on GPQA and SVAMP(86.9% is the Qwen2.5-7B-Instruct result reported in CARFT (Zhu et al., 2025)), with three random seeds and a maximum generation length of 1,024 tokens. Table R1 will report accuracy as mean ± standard deviation across the three seeds, including the no-memory baseline, Native MemGen, the combined system, and component ablations.

Table R1. Supplementary accuracy (%) under matched evaluation settings.

BackboneDatasetBaselineMemGenCSMC + DAR + RSATCSMC + DARCSMC + RSATDAR + RSATCSMCDARRSAT
Qwen2.5-7B-InstructGPQA25.25 ± 0.0025.59 ± 2.2823.91 ± 1.2728.11 ± 2.7827.78 ± 0.8722.56 ± 1.1727.78 ± 1.5224.07 ± 0.7725.76 ± 2.20
Qwen2.5-7B-InstructSVAMP86.982.33 ± 6.3588.44 ± 3.2988.78 ± 2.3687.78 ± 3.2986.67 ± 7.5486.67 ± 2.3384.22 ± 5.4084.78 ± 3.67

Response to W1:

We agree that a no-memory baseline is necessary to determine whether latent-memory augmentation benefits the frozen reasoner. Comparisons against MemGen alone cannot establish this.

We add plain-CoT baselines without latent memory for Qwen2.5-7B on GPQA and SVAMP. The supplementary comparisons use a maximum generation length of 1,024 tokens. Table R1 reports accuracy as mean ± standard deviation over three random seeds. For Qwen2.5-7B-Instruct on GPQA, the frozen reasoner with plain CoT prompting and no latent memory achieves a mean accuracy of 25.25%. After removing overlapping training examples and rerunning all training stages, CSMC achieves 29.80%, compared with 25.59% for MemGen (Table R2). These results provide a direct comparison with the unaugmented reasoner in this setting.

Response to W2:

We thank the reviewer for highlighting the potential influence of answer formatting and extraction. The official GSM8K accuracy of 91.6% and our reported MemGen accuracy of 73.77% were obtained under different evaluation protocols, so the gap alone does not establish an evaluation error. Nevertheless, we agree that formatting and extraction may affect measured accuracy and may have contributed to the reported gains.

Due to limited computational resources, we have so far rerun the experiments using the revised answer format only on GPQA and SVAMP, the smaller benchmarks in our evaluation, to investigate this concern. All methods use the same answer format requirements, extraction rules, and generation budget. We will document these settings explicitly. All supplementary runs retain a generation limit of 1,024 tokens. We consider accuracy within a shared token budget a meaningful measure of reasoning effectiveness: solving more problems under that constraint is a useful improvement.

Response to W3:

We agree that results from a single run provide insufficient evidence for the reliability of small improvements. We use the evaluation with three random seeds and will report the mean and standard deviation for each configuration in Table R1.

We also agree that a difference of 1 to 2 percentage points on the 198 GPQA questions corresponds to only a few answers. We will interpret such differences cautiously and discuss the observed variability rather than treating an isolated single-run gain as conclusive evidence.

AuthorsOpenReview ↗

Response to Reviewer oqT2 [Part 2/3]

Response to W4:

Our modifications intentionally build on MemGen, and we appreciate the need to make the contribution boundary clearer. The contribution we aim to establish is a controlled examination of how supervision and memory capacity, extraction depth, and recall timing affect latent-memory augmentation of frozen reasoners.

Regarding CoT supervision, we use distillation to investigate what information the Weaver should learn to generate useful latent memory. In MemGen, this choice is largely determined by the dataset’s native supervision targets, leaving open whether brief rationales provide sufficient reasoning information for memory generation. We revisit this choice by supervising the Weaver with more detailed intermediate reasoning steps and their logical connections. With the architecture unchanged and the reasoner frozen, our experiments provide evidence that richer reasoning supervision can improve the usefulness of the generated memory. This offers empirical guidance on what to remember, making the content of memory supervision an explicit design choice.

Regarding compression, our goal is to retain useful reasoning information with fewer injected memory vectors. The compression configurations reduce the number of injected vectors by 75% on GSM8K, SVAMP, and StrategyQA, and by 50% on GPQA. These results reflect a tradeoff between accuracy and the injected memory budget, with effects on accuracy varying across configurations.

Regarding DAR, we also explored a layer near the output, indexed as -2, in our earlier experiments and observed performance similar to that of the last layer. Our motivation is that representations near the output may be more closely tied to the semantics of immediate token predictions, whereas intermediate layers may retain more abstract structures useful for reasoning. To investigate what information should be retained as latent memory, we therefore included intermediate readout as an alternative. The results suggest that intermediate layers may be a more suitable source of latent memory, consistent with our hypothesis that retaining abstract reasoning information can benefit the frozen reasoner.

Response to W5:

We agree that selecting the default entropy band using the same evaluations on which its performance is reported can introduce selection bias.

Nevertheless, the original results provide additional context: all four evaluated bands yield positive average gains over punctuation gating. With the upper cutoff fixed at P95, the lower cutoffs P75, P80, P85, and P90 yield mean gains of 1.11, 2.05, 2.02, and 2.48 percentage points, respectively, across the 16 configurations. The default band [P80, P95) was selected by aggregate Borda ranking, rather than by the highest mean accuracy; [P90, P95) achieves the largest mean gain. These results show that the positive average gains are not confined to the selected default band.

We will clarify the selection procedure and present [P80, P95) as a recommended entropy range based on the evaluated configurations, rather than a uniquely optimal or universally applicable choice.

Response to W6:

We agree that the separate comparisons in the original submission cannot determine whether the components remain beneficial when combined.

We add the complete CSMC + DAR + RSAT system and ablations that remove one component at a time, as shown in Table R1. Comparison with MemGen evaluates the overall change, while the ablations assess whether each component contributes when the other two are present.

Response to W7:

We understand that the missing no-memory baseline and the absence of repeated runs are central to the reviewer’s assessment. We agree that these omissions limit the conclusions supported by the original evaluation.

The supplementary comparisons in W1 and W3 directly address these concerns, alongside the corrected GPQA experiments described in Q3. We will base the revised claims on these results and explicitly state their coverage and limitations.

AuthorsOpenReview ↗

Response to Reviewer oqT2 [Part 3/3]

Response to Q1:

Table R1 reports the complete CSMC + DAR + RSAT configuration and component ablations, allowing us to examine whether the components remain useful in combination.

Regarding the concern about selecting the RSAT band on an independent validation split, please see our response to W5, where we discuss the selection procedure and potential selection bias.

Response to Q2:

The requested results for plain CoT prompting without memory are reported in Table R1. The reasoner’s parameters remain frozen, and the baseline uses the same final answer format, extraction rules, and generation limit of 1,024 tokens as the methods using latent memory.

Response to Q3:

Unparseable answers are counted as incorrect and remain in the denominator when computing accuracy.

We acknowledge the concern about potential overlap between the training and test data in the original GPQA setting. Our original implementation followed the data configuration used in our MemGen reproduction, and the compared methods shared that configuration. However, using the same data setup across methods does not eliminate leakage or establish generalization to unseen questions.

To address this issue, we are rerunning the experiments after removing training examples that overlap with the GPQA evaluation questions. We will explicitly document the source dataset, exclusion procedure, number of removed examples, and training stages rerun.

We will complete these experiments as soon as possible and post the results within the next one to two days. The updated results will be reported in Table 2.

Response to Q4:

The supplementary evaluation uses three random seeds, and each entry in Table R1 will report mean accuracy and standard deviation.

Because of computational constraints, this evaluation covers Qwen2.5-7B on GPQA and SVAMP. It does not yet cover every backbone–dataset configuration in the original main tables.

Reviewer_xixROpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This fabulous paper presents an approach to make use of the latent (trained and frozen) memory to be leveraged for testing time reasoning in conjunction with any Lora model used. The proposal provides (a) what vignettes to use for the CoT training such that it is no overfit, high performance, and low-data for compact representation (clever), (b) which intermediate layer to reiterative or extract from (this prevents saving correlated bulks), and (c) how to trigger the extraction as a mechanism to get to (a) and (b). The idea of not lumping everything into the last layer and having a low-latent dimensional space for a set of vignettes to be used to train for the CoT is superbly good. It beats the older rules encoding in the symbolic manner. Used it and seen the impact in my own works. But have not seen in this context! Much needed approach instead of relying on unproven verifications or "rules."

Critical strengths

Providing an efficient mechanism to pre-train the latent memory using a set of vignettes at the time of the CoT training and freezing this latent memory that will be used later. Done well, the results can be excellent for specific tasks.

Critical weaknesses

One of the bigger challenges here is as the CoT training vignettes have to be developed by another mechanism and if these are super large, the performance could be a bit problematic. But the idea is really useful compared to clumping to the last layer. This approach allows better performance in conjunction with any LoRa. Surprisingly, if the vignettes are done well, the LoRa really does not add much to the performance in my experience.

Questions for authors

How well you train the frozen memory is the bigger bottleneck. What are your thoughts?

3 comments
AuthorsOpenReview ↗

Response to Reviewer xixR

We thank the reviewer for the encouraging assessment and the thoughtful comments on CoT supervision and memory-training quality.

Response to W1:

We agree that generating detailed CoT targets introduces additional data-construction cost, and longer targets increase Weaver training cost. In our setup, these targets are constructed offline and used only for Weaver supervision. At inference time, the teacher is not queried, and its full textual rationales are not inserted into the reasoner’s context.

Our compression component addresses the memory budget used during inference. Specifically, CSMC reduces prompt/inference memory lengths from 16/8 to 4/2 on GSM8K, SVAMP, and StrategyQA, and to 8/4 on GPQA. This corresponds to 75% and 50% fewer injected memory vectors, respectively. Under matched invocation patterns, the reasoner has fewer latent-memory positions to store and attend to, reducing the storage and attention overhead associated with these positions.

To clarify the role of LoRA, it is used to adapt the Weaver and Trigger in our framework. The Weaver is frozen after SFT, and the base reasoner remains frozen throughout training. Thus, our experiments study improvements to latent-memory generation without additional fine-tuning of the reasoner.

Response to Q1:

We agree that training the Weaver well is a key bottleneck, particularly because it is frozen after SFT and cannot be further updated during Trigger optimization. Our results support this view: with the LoRA configuration unchanged, replacing native supervision with teacher-generated CoT improves accuracy in 15 of 16 configurations, with a macro-average gain of 9.30 percentage points (Table 1).

We would further distinguish between learning useful information and making that information useful to the frozen reasoner. Memory quality depends on both what the Weaver learns and how its representations are accessed. For example, our DAR experiments retain native supervision while changing the readout depth; intermediate-layer readout improves macro-average accuracy by 4.50 percentage points over final-layer readout (Table 2). This suggests that the effectiveness of the memory pathway depends on representation selection as well as supervision.

Our perspective is therefore that improving Weaver training provides an essential foundation, while readout depth and recall timing are additional factors governing how effectively the learned information supports reasoning. This motivates our What–Where–When framework: What examines the information learned and retained, Where examines the representations extracted, and When examines the opportunities to use them.

Thank you for the clarifications. I see other reviewers's comments and clearly they have some concerns. Hopefully you can address them very well. I feel the main challenge will be related to how much of the vignettes you will have to choose. Not so surprisingly, the current systems can start extracting "rules" if you play adequate vignettes. These are not classical rules. Anyway, I am satisfied with your answers.

Thank you for your follow-up and helpful observations. We agree that the amount and selection of reasoning examples, and the reusable patterns learned from them, are valuable directions for further investigation. We have also added supplementary experiments and clarifications to the revised manuscript to address the concerns raised by the other reviewers. We appreciate your constructive feedback throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.