acceptodds
Under review as a conference paper at ICLR 2027

Balancing Memory Pathways: Analyzing and Improving Memory Utilization in LLMs

Abstract

Recurrent–attention hybrid language models (LMs), which interleave attention and recurrent layers, are increasingly used to combine the efficiency of the recurrent layers with the strong performance of attention layers. Prior work suggests that attention and recurrent layers offer complementary pathways to use past information: attention supports precise memory recall from earlier tokens, while recurrent layers support consolidation of disparate information over long spans. However, we observe that simply having access to both pathways does not mean that hybrid LMs are effectively using them. When we analyze how much the models rely on each pathway, we find a substantially greater reliance on attention than on the recurrent state. Standard supervised fine-tuning improves overall performance but does not improve how the two memory pathways are coordinated: the model becomes more reliant on information propagated by attention layers, while its use of information propagated by recurrent layers remains limited. To encourage better coordination between the two memory pathways, we add an auxiliary loss that limits attention’s access to earlier context while the recurrent state propagates through the full sequence. By backpropagating the auxiliary loss through the recurrent layers, this objective encourages the model to retain information from earlier context in recurrent state and use it alongside information accessed through attention. We observe increased use of the recurrent memory pathway and improved overall performance. Crucially, this imbalance and the benefit of our auxiliary loss generalize: they apply to multiple recurrent-attention LMs in question-answering and agentic tasks, as well as to attention-based LMs that combine different forms of memory. Together, our findings show that simply providing multiple memory pathways does not ensure their effective use, and that targeted supervision is needed to coordinate them.

Primary area:
foundation or frontier models, including LLMs
Keywords:
hybrid memory, language models, recurrent-attention models, memory coordination

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7JtMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies recurrent-attention hybrid language models and shows that standard supervised fine-tuning (SFT) further shifts memory use toward attention while leaving the recurrent pathway underutilized. It proposes a simple auxiliary loss, in which an auxiliary forward pass masks attention access to past context so that past information must reach the current prediction through recurrent state. This intervention improves long-context QA and agentic task performance on Qwen3.5-4B and Nemotron-H-4B, and the authors further show that the same imbalance and auxiliary-supervision benefit generalize to attention-based LMs with textual and latent memory.

Critical strengths
  • The empirical analysis of pathway imbalance is clear and useful: the paper demonstrates that providing both attention and recurrent pathways does not imply balanced use, and that standard SFT can reinforce attention reliance. This is a consequential finding for hybrid LM design and fine-tuning.
  • The proposed auxiliary loss is simple but effective across models and task families, including QA, TextWorld/BabyAI, failure-recovery settings, and textual/latent memory. The breadth of validation makes the intervention more than a single-benchmark artifact.
Critical weaknesses
  • The comparison is selectively narrow with respect to contemporaneous 2025–2026 hybrid-LM fine-tuning interventions. The paper compares only against standard SFT and internal ablations, omitting methods on the same model family and tasks, such as S0 Tuning (+23.6 pp pass@1 on Qwen3.5-4B), selective GDN freezing, and component-aware LoRA placement. This leaves unclear whether L_rec offers an advantage over existing coordination strategies, which is consequential for the paper’s central claim that targeted supervision is needed to coordinate memory pathways.
  • The contribution is primarily empirical. Masking attention in an auxiliary forward pass to force reliance on another memory pathway is close to existing masking/regularization techniques. The paper should more sharply distinguish its contribution from prior memory-coordination, retrieval-aware distillation, and component-specific fine-tuning methods, and should clarify which design choices are newly introduced versus inherited.

Young, J. (2026). S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models. arXiv. https://arxiv.org/abs/2604.01168

Questions for authors
  • How does L_rec compare against S0 Tuning, selective layer freezing, and component-aware LoRA placement under an identical protocol on Qwen3.5-4B? Since S0 Tuning reports substantially larger absolute gains on the same model, what evidence supports the claim that recurrent auxiliary supervision is the preferable coordination mechanism?
  • How do the gains compare at strictly equal wall-clock/FLOPs and under full fine-tuning or larger hybrid models, rather than LoRA on 4B models only?
  • Could the gains be explained mainly as a regularization effect rather than improved recurrent-pathway use? What evidence isolates pathway utilization from generic regularization?
7 comments

We thank the reviewer for their thoughtful comments and for recognizing the clarity of our analysis of the pathway imbalance, the simplicity and effectiveness of our auxiliary loss, and the breadth of our experiments.

We provide our detailed responses below.

W1:The comparison is selectively narrow with respect to contemporaneous 2025–2026 hybrid-LM fine-tuning interventions. The paper compares only against standard SFT and internal ablations, omitting methods on the same model family and tasks, such as S0 Tuning (+23.6 pp pass@1 on Qwen3.5-4B), selective GDN freezing, and component-aware LoRA placement. This leaves unclear whether L_rec offers an advantage over existing coordination strategies, which is consequential for the paper’s central claim that targeted supervision is needed to coordinate memory pathways.

Q1: How does L_rec compare against S0 Tuning, selective layer freezing, and component-aware LoRA placement under an identical protocol on Qwen3.5-4B? Since S0 Tuning reports substantially larger absolute gains on the same model, what evidence supports the claim that recurrent auxiliary supervision is the preferable coordination mechanism?

Thank you for the suggestion. As suggested, we conduct experiments comparing with suggested baselines including S0-Tuning[1] and component-aware LoRA placement[2]. As shown in table below, we evaluate overall performance with previous context passed through both pathways (Full) and use the attention-only and recurrent-only setups as ablations, passing previous context through only the corresponding pathway. We observe that training with our auxiliary loss (Ours) shows strongest performance on both question answering and agentic tasks. Also, we observe that the baselines tend to underuse the recurrent pathway as we observe with standard SFT, whereas ours show balanced use of both pathways. We have included the results in Appendix C.3.

In more detail, we analyzed why S0-Tuning did not yield performance gains in our settings as those reported in their paper. We identified two possible explanations. First, the paper experimented over a small subset of HumanEval (48 instances) for training where they observed the model they compared with (standard LoRA) tended to overfit whereas S0-Tuning avoids this issue. Our training set is much larger (12k). Second, the dataset they used (HumanEval) is much shorter compared to our task (our QA is around 80 times longer on average). As S0-Tuning modifies only the initial recurrent state, its influence may diminish over these longer sequences.

Also, we were unable to identify the paper on selective GDN freezing mentioned in the review. We would appreciate it if the reviewer could share the title or citation so that we can evaluate the method.

MethodQA FullQA attention-onlyQA recurrent-onlyTextWorld (Quest) FullTextWorld (Quest) attention-onlyTextWorld (Quest) recurrent-only
SFT34.625.05.078.552.04.0
S0-Tuning30.320.94.150.038.52.0
LoRA (attn)34.825.53.767.550.01.5
LoRA (rec)36.025.04.267.553.01.5
Ours38.523.722.880.553.046.5

W2:The contribution is primarily empirical. Masking attention in an auxiliary forward pass to force reliance on another memory pathway is close to existing masking/regularization techniques. The paper should more sharply distinguish its contribution from prior memory-coordination, retrieval-aware distillation, and component-specific fine-tuning methods, and should clarify which design choices are newly introduced versus inherited.

Q3: Could the gains be explained mainly as a regularization effect rather than improved recurrent-pathway use? What evidence isolates pathway utilization from generic regularization?

Thank you for your question and suggestions. Since the question raises several points, we have organized our response into bullets.

  • Our Contribution

We would like to highlight that our contributions are mainly on threefold:

  1. In Section 4, we show that hybrid LMs still underuse the recurrent pathway after standard SFT.
  2. In Section 5, to use both pathways well, we introduce an auxiliary loss that encourages use of the recurrent pathway.
  3. In Section 6, we analyze how the auxiliary loss helps to learn from past trajectories and show that the finding generalizes to attention only models with textual and latent memory.

To our knowledge, our work is the first to show that hybrid LMs tend to underuse the recurrent pathway even after standard SFT (Section 4) and to propose a method specifically designed to address this imbalance. As the reviewer mentioned, the auxiliary loss in Section 5 builds on an existing masking mechanism, which we also mentioned in the paper as “commonly used to control information flow in LMs”. Our contribution is to identify this imbalance and design an auxiliary loss that uses this mechanism to encourage greater use of the recurrent pathway.

  • Comparison with existing masking / regularization techniques

To evaluate whether generic regularization or masking can account for these gains, we conduct experiment with comparison with regularization and masking baselines:

  1. R-Drop [3], a regularization method that passes the same input through the model twice with different dropout samples and minimizes the symmetric KL divergence between the two output distributions.
  2. Input token dropout [4], a regularization method that replaces a random subset of input tokens.
  3. Random attention masking [5], a masking method that randomly masks attention entries.

As shown in table below, on the question answering task, all baselines improve accuracy (Full) over standard SFT, but the pathway imbalance persists, recurrent-only showing much lower performance compared to attention-only. Also, ours with auxiliary loss show highest performance (Full). These results support that specifically encouraging the use of the recurrent pathway contributes to the gain rather than generic regularization. We include more details in Appendix C.6.

MethodFullattention-onlyrecurrent-only
SFT34.625.05.0
R-Drop35.326.76.5
Input token dropout37.027.46.5
Random Attention Masking35.930.87.1
Ours38.523.722.8
  • Evidence of improved recurrent pathway use

Along with our analysis of recurrent-only, we also provide evidence in Sections 5.2 and 6 that better use of the recurrent pathway contributes to improved performance.

In Section 5.2, we show that the gains are larger in longer-context settings for both QA and agentic tasks. This pattern is consistent with our analysis in Section 2 and with prior work suggesting that the recurrent pathway is particularly useful for aggregating information across long contexts.

In Section 6, we observe that the model trained with the auxiliary loss makes better use of previous trajectories, especially as more trajectories are provided. Also, it shows larger gains on multi-evidence questions (Fig. 7), for which we found the recurrent pathway to be particularly helpful (Fig. 3).

  • Relation to prior lines of work (memory coordination, retrieval-aware distillation, and component-specific fine-tuning)

Retrieval-aware distillation [6] builds a hybrid by keeping only the few attention heads needed for precise retrieval and converting the rest into recurrent heads. The paper focuses on a different question from our work: what is new way of designing hybrid LMs, whereas our work focuses on making the existing hybrid LMs better use both pathways.

Component-specific fine-tuning [2] restricts which components are updated (e.g., only the attention or recurrent layers) whereas ours update all components whereas change how information flows. Thus, component-specific fine-tuning does not balance the utilization as we observed in W1.

Memory coordination: Our work focuses on memory coordination between the two pathways of hybrid LM. To our knowledge, no prior work has shown that the bias towards attention persists under standard SFT, or that correcting it to coordinate the two pathways better improves performance.

We discussed representative work on each topic above but if the reviewer had other works in mind, we would be glad to discuss them.

Q2: How do the gains compare at strictly equal wall-clock/FLOPs and under full fine-tuning or larger hybrid models, rather than LoRA on 4B models only?

  • Equal wall-clock/FLOPs

As described in Section 3 (Training), each optimization step with the auxiliary loss requires additional computation. Thus, to account for this cost, we use half as many optimization steps as standard SFT.

  • Full fine-tuning or larger hybrid models

Thank you for the suggestion. As suggested, we ran experiments with full fine-tuning (Full FT) on Qwen3.5-4B and LoRA fine-tuning on Qwen3.5-9B. We observe similar trends in both settings: (1) with standard SFT, the models tend to underuse the recurrent pathway; (2) adding auxiliary loss (Ours) leads to more balanced use of both pathways and improves performance. We have updated in Appendix C.7.

MethodQA FullQA attention-onlyQA recurrent-onlyTextWorld (Quest) FullTextWorld (Quest) attention-onlyTextWorld (Quest) recurrent-only
SFT (4B, LoRA)34.625.05.078.552.04.0
Ours (4B, LoRA)38.523.722.880.553.046.5
SFT (4B, Full FT)39.528.14.478.552.59.0
Ours (4B, Full FT)40.423.621.082.056.039.0
SFT (9B, LoRA)43.728.99.384.059.07.0
Ours (9B, LoRA)45.126.723.285.058.547.0

[1] S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models

[2] Where Should LoRA Go? Component-Type Placement in Hybrid Language Models

[3] R-Drop: Regularized Dropout for Neural Networks

[4] Data Noising as Smoothing in Neural Network Language Models

[5] DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks

[6] Retrieval-Aware Distillation for Transformer-SSM Hybrids

Reviewer_nXNFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The article examines how the recurrent-attention hybrid language models make use of their two different pathways for maintaining information about the past. By means of interventions at the segment level, such as masking the cross-segment attention or resetting the recurrent state at the segment boundary, the paper concludes that the models primarily depend on the attention mechanism and that SFT reinforces this dependence. It introduces an auxiliary loss, L_rec: this involves a second forward pass in which attention to the past context is masked, and it is trained using the same next-token loss. When using LoRA finetuning, this approach leads to an improvement in average QA accuracy across six datasets, mainly on the long context datasets, and also increases the success rates in TextWorld and BabyAI.

Critical strengths

It offers a simple method that leads to steady improvements. L_rec does not require any change to its architecture, is about equal in terms of computing resources, and proves beneficial for the models in both the QA tasks and in the two agentic environments. The gains obtained with L_attn are smaller, which is in the direction expected by the paper's narrative.

Critical weaknesses

The diagnostics do not provide clear support for the idea of an imbalance. The two interventions are not equally disruptive since masking all cross-segment attention also eliminates the BOS/attention sink tokens and results in large position offsets, something which could cause the model to fail on its own. Resetting the recurrent state is more in line with in distribution behaviour, and the short conv state is not dealt with. Without a quetion only baseline, the fact that recurrent-only accuracy is about 5% cannot be interpreted as meaning that the recurrent state carries very little useful information. Moreover, the retention ratios (in Figures 3 and 5) are affected by floor effects and by the way the dataset is composed.

The gains are not attributed to the proposed mechanism. Higher recurrent-only accuracy after L_rec is expected, since L_rec trains in exactly that condition. So reliance on attention roughly halves, contradicting the claim that attention use is preserved. L_rec may also act as a generic regularizer. L_attn is not a matched control, because its auxiliary view is much closer to the full pass.

Questions for authors

What is the closed book accuracy for each dataset? And how do the results for models using only recurrent connections change if the BOS or sink tokens stay visible?

Under attention only, is the short convolution state also reset at the boundary?

On either the validation data or the test data was λ chosen? How does halving the number of steps interact with the setting of 5 epochs together with early stopping? Why does using just L_attn improve the accuracy of Qwen's DROP by 8.3 points?

In the agentic setting, what is the method used for defining segments and which of the tokens have their loss calculated during the auxiliary pass?

5 comments

We thank the reviewer for thoughtful comments and recognizing the simplicity of our approach and its performance gains on QA tasks and in two agentic environments.

We provide our detailed responses below.

W1 The diagnostics do not provide clear support for the idea of an imbalance. The two interventions are not equally disruptive since masking all cross-segment attention also eliminates the BOS/attention sink tokens and results in large position offsets, something which could cause the model to fail on its own. Resetting the recurrent state is more in line with in distribution behaviour, and the short conv state is not dealt with. Without a quetion only baseline, the fact that recurrent-only accuracy is about 5% cannot be interpreted as meaning that the recurrent state carries very little useful information. Moreover, the retention ratios (in Figures 3 and 5) are affected by floor effects and by the way the dataset is composed.

Q1: What is the closed book accuracy for each dataset? And how do the results for models using only recurrent connections change if the BOS or sink tokens stay visible?

Thank you for your question and suggestions. Since the question raises several points, we have organized our response into bullets.

  • Adding sink tokens

As the reviewer suggested, we ran experiments with sink tokens visible and observed that the recurrent-only scores remain similarly low. Since Qwen3.5 inputs do not begin with a dedicated BOS token, we used the chat template tokens, which are included at the beginning of every input. Our measurements also confirm that they act as the sink tokens. We have included the result in Appendix C.4.

recurrent-onlyBaseSFTSFT+REC (Ours)
masked5.15.022.8
visible5.15.323.4
  • Question-Only baseline

As the reviewer suggested, we conduct a question-only baseline. As shown in table below, for the base and SFT models, recurrent-only accuracy falls below question-only, whereas with our auxiliary loss (SFT+REC), it exceeds question-only. These results support our finding in Section 4 that the model makes limited use of the recurrent pathway after standard SFT and in Section 5 that the auxiliary loss helps the model make better use of this pathway. We included more detailed results in Appendix C.5.

SettingBaseSFTSFT+REC (Ours)
Attention-only20.225.023.7
Recurrent-only5.15.022.8
Question-only10.912.211.8

W2: The gains are not attributed to the proposed mechanism. Higher recurrent-only accuracy after L_rec is expected, since L_rec trains in exactly that condition. So reliance on attention roughly halves, contradicting the claim that attention use is preserved. L_rec may also act as a generic regularizer. L_attn is not a matched control, because its auxiliary view is much closer to the full pass.

Thank you for your question and suggestions. Since the comment raises several points, we have organized our response into bullets.

  • Evidence that the gains come from the recurrent pathway.

Along with our analysis of recurrent-only, we also provide evidence in Sections 5.2 and 6 that better use of the recurrent pathway contributes to improved performance.

In Section 5.2, we show that the gains are larger in longer-context settings for both QA and agentic tasks. This pattern is consistent with our analysis in Section 2 and with prior work suggesting that the recurrent pathway is particularly useful for aggregating information across long contexts.

In Section 6, we observe that the model trained with the auxiliary loss makes better use of previous trajectories, especially as more trajectories are provided. Also, it shows larger gains on multi-evidence questions (Fig. 7), for which we found the recurrent pathway to be particularly helpful (Fig. 3).

  • L_rec may act as a generic regularizer

To evaluate whether generic regularization can account for these gains, we conduct experiment with comparison with two regularization baselines:

  1. R-Drop [1], a regularization method that passes the same input through the model twice with different dropout samples and minimizes the symmetric KL divergence between the two output distributions.
  2. Input token dropout [2], a regularization method that replaces a random subset of input tokens.

As shown in table below, we evaluate overall performance with previous context passed through both pathways (Full) and use the attention-only and recurrent-only setups as ablations, passing previous context through only the corresponding pathway. On the question answering task, as shown in table below, both regularization baselines improve accuracy (Full) over standard SFT, but the pathway imbalance persists, recurrent-only showing much lower performance compared to attention-only. Also, ours with auxiliary loss show highest performance (Full). Moreover, in the paper, we conduct an experiment showing that encouraging the recurrent pathway yields large gains compared to encouraging the attention pathway. Together, these results support that specifically encouraging the use of the recurrent pathway contributes to the gain rather than generic regularization. We include more details in Appendix C.6.

MethodFullattention-onlyrecurrent-only
SFT34.625.05.0
R-Drop35.326.76.5
Input token dropout37.027.46.5
Ours38.523.722.8
  • Matching L_attn and L_rec

As the reviewer mentioned, the auxiliary pass for L_attn is closer to the full pass than L_rec. However, this difference reflects the imbalance in pathway use that we identified in Section 4, which motivates adding the auxiliary loss. We therefore retain this difference in the comparison. Please note that all other training settings are the same and we vary only which pathway can access earlier context during the auxiliary pass.

  • Reliance on attention roughly halves

We would appreciate it if the reviewer could clarify which result they are referring to. In Figure 6, attention-only accuracy remains close to the standard SFT result (23.7% with our method vs. 25.0% with standard SFT).

Q2 Under attention only, is the short convolution state also reset at the boundary?

Yes. In the attention-only condition, both the recurrent state and the short-convolution state are reset at the boundary so that information from the earlier segment can reach the current segment only through attention.

Q3 On either the validation data or the test data was λ chosen? How does halving the number of steps interact with the setting of 5 epochs together with early stopping? Why does using just L_attn improve the accuracy of Qwen's DROP by 8.3 points?

We selected λ based on validation performance. The original figure reported test performance to show that training with the auxiliary loss consistently outperformed standard SFT across all tested values of λ\lambda. We have updated figure and Appendix C.1 to be more clear.

As described in Section 3, we halve the number of optimization steps for objectives with an auxiliary loss to match the total training compute of standard SFT, since each step requires additional computation.

One possible explanation for the 8.3 point improvement on DROP with L_attn is that many DROP questions require precise recall, which our analysis in Section 2 identifies as a strength of the attention pathway. However, our further analysis did not reveal a clear pattern that could explain the gain.

Q4 In the agentic setting, what is the method used for defining segments and which of the tokens have their loss calculated during the auxiliary pass?

Each turn forms one segment and in the auxiliary pass, attention is restricted to the current turn, so the goal, past observations, and past actions can reach the current turn only through the recurrent state.

[1] R-Drop: Regularized Dropout for Neural Networks

[2] Data Noising as Smoothing in Neural Network Language Models

Reviewer_afduOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to analyze the effects of interleaving attention and recurrent layers in language models based on the investigation that attention supports precise memory recall from earlier tokens, while recurrent layers support consolidation of disparate information over long spans. The author further proposes a methodology to better coordination between the two memory pathways. The experiments are conducted on two backbone models on several datasets. However, there are no baselines compared in the manuscript.

Critical strengths

The motivation is indicated clearly, and the experimental results can support the claim to some extent.

Critical weaknesses
  1. The example in Figure 1 used to motivate the scenario is not supportive; i.e., how can open the safe run into 100 turns and how is it relevant to the proposed methods and targeted task?

  2. The results shown in Figure 2 and Figure 3 are intuitive, while there are no distinguish compared to existing studies.

  3. In terms of existing studies and the experimental setup, there are no state-of-the-art baselines used in comparions. The main results shown at Table 1 only contain different figurations of the proposed methods but not existing approaches. Thus, it is not convinced and sufficient to support the effectiveness.

Questions for authors

Please add enough comparative baselines and distinguish the proposed methods with existing studies. Besides, please explain clearly how memory relates to hybrid LMs.

4 comments

We thank the reviewer for thoughtful comments and recognition of clear motivation and the experimental results supporting our findings.

We provide our detailed responses below.

W1 The example in Figure 1 used to motivate the scenario is not supportive; i.e., how can open the safe run into 100 turns and how is it relevant to the proposed methods and targeted task?

Thank you for the question. The example in Figure 1 is drawn from an actual TextWorld episode [1]. TextWorld is a partially observable environment where each observation reveals only part of the current state (e.g., room, inventory).

Thus, to choose the right action and solve the task, the agent needs to aggregate evidence across earlier observations and recall specific details from them [2][3]:

  1. It needs to aggregate and reason over scattered evidence (green) such as which exits are blocked and which containers have been searched. When this fails, the agent can get stuck repeating actions and fail to solve the task.
  2. It needs to recall specific past observations (orange) precisely, such as where it saw a locker several turns earlier, so it can return to it once it finds the key.

These two abilities align with our findings in Section 2, which suggest that the recurrent pathway helps integrate evidence across the context, while the attention pathway helps recover specific details from earlier observations. Thus, the agent needs to use both pathways well to solve the task.

The ‘up to 100 turns’ refers to the maximum number of turns allowed per TextWorld episode. In Fig1, we show an example of an unsuccessful trajectory that ended when the agent reached this limit without opening the safe.
The oracle solves the episode in far fewer turns, but because TextWorld is partially observable, the agents often reach this limit because they get stuck repeating actions after failing to aggregate earlier observations or recall a specific detail as it cannot see full observation and solve the issue through exploration.
However, as mentioned in section 5.2, the average steps for successful trajectories is 40.2 and 13.8 steps for TextWorld.

We have updated Figure 1 and its caption and added examples of successful and unsuccessful trajectories in Appendix D.

| W2: The results shown in Figure 2 and Figure 3 are intuitive, while there are no distinguish compared to existing studies.

Thank you for the question. Fig2 was conducted to show that the pattern from prior findings persists in recent hybrid LM and datasets we used. As these observations serve as the starting point for our analyses in Sections 4, 5, and 6, and prior studies conducted experiments on smaller and earlier models, we wanted to first show that the observation still holds with the models and datasets we are using. We have also noted in the paper (“Figure 2 shows a pattern consistent with prior findings”).

In contrast, Figure 3 is our own contribution where we conduct a finer-grained analysis by question type, which forms the basis of our later analysis of how much each pathway is actually used.

Furthermore, as its title indicates, Section 2 is focused on background and related work. Our core contributions are in Section 4, 5, and 6:

  • In Section 4, we show that hybrid LMs still underuse the recurrent pathway after standard SFT.
  • In Section 5, to use both pathways well, we introduce an auxiliary loss that encourages use of the recurrent pathway
  • In Section 6, we analyze how the auxiliary loss helps to learn from past trajectories and show that the finding generalizes to attention only models with textual and latent memory.

We would also appreciate any references to specific existing studies the reviewer had in mind for comparison.

W3: In terms of existing studies and the experimental setup, there are no state-of-the-art baselines used in comparions. The main results shown at Table 1 only contain different figurations of the proposed methods but not existing approaches. Thus, it is not convinced and sufficient to support the effectiveness.

Q1: Please add enough comparative baselines and distinguish the proposed methods with existing studies. Besides, please explain clearly how memory relates to hybrid LMs.

Thank you for the question. To our knowledge, our work is the first to show that hybrid LMs tend to underuse the recurrent pathway even after standard SFT (Section 4). Thus, there was no prior work that focused to address this imbalance.

To address the reviewer’s concern about the performance of our auxiliary loss, we ran experiments with training methods proposed in prior work to improve overall model performance. We tested S0-Tuning [4] which adapts the model to a target task by learning the initial state of each recurrent layer, and component-aware LoRA [5] which trains LoRA in either only the attention or recurrent layers. We evaluate overall performance with previous context passed through both pathways (Full) and use the attention-only and recurrent-only setups as ablations, passing previous context through only the corresponding pathway. As shown in the table below, our method achieves the best performance on both tasks in the Full setting. Also, the imbalance between two pathways persists with both prior methods, for which the recurrent-only ablation performs substantially worse than the attention-only ablation. We have included the results in Appendix C.3.

MethodQA FullQA attention-onlyQA recurrent-onlyTextWorld (Quest) FullTextWorld (Quest) attention-onlyTextWorld (Quest) recurrent-only
SFT34.625.05.078.552.04.0
S0-Tuning30.320.94.150.038.52.0
LoRA (attn)34.825.53.767.550.01.5
LoRA (rec)36.025.04.267.553.01.5
Ours38.523.722.880.553.046.5

We would also like to highlight that the hybrid LMs we use (Qwen 3.5 and Nemotron-H) are strong and widely used base models.

Moreover, we would appreciate any suggestions for specific methods or papers that the reviewer considers relevant for comparison.

[1] TextWorld: A Learning Environment for Text-based Games

[2] TALES: Text Adventure Learning Environment Suite

[3] Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty

[4] S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models

[5] Where Should LoRA Go? Component-Type Placement in Hybrid Language Models

Dear Reviewer afdu,

Thank you for your thoughtful review and valuable feedback. With about three days remaining in the discussion period, we wanted to follow up to ask whether our rebuttal has addressed your main concerns. Our response includes an updated Figure 1 with a more detailed explanation, clarification of our work’s novelty, and comparisons with other training methods.

We would be happy to discuss any remaining concerns or provide further clarification. Thank you again for your time and consideration.

Dear Reviewer afdu,

As the discussion period ends on November 18 (AoE), we would appreciate if you could review our responses and let us know whether we have addressed your concerns. We would also be happy to clarify any remaining points. Thank you for your time!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.