acceptodds
Under review as a conference paper at ICLR 2027

User Queries Say More with Less: Query-Anchored, LLM-Aware Response Selection for Long-Term Conversational Memory

Abstract

In long-term conversational memory, historical user queries and LLM responses exhibit a pronounced asymmetry: user queries are concise and information-dense, whereas LLM responses are substantially longer but often contribute limited new information. We find that historical user queries serve as the primary memory basis, while LLM responses are largely compressible but conditionally useful, with their utility depending on relevance to the current query and the needs of the downstream LLM. Based on these findings, we propose QARS (Query-Anchored Response Selection), a context selection framework that retains historical user queries as memory anchors and selectively incorporates historical LLM responses. It first filters LLM response candidates by their semantic relevance to the current query, thereby narrowing the candidate set; it then performs downstream-LLM-aware selection over the remaining candidates using trained discriminators conditioned on a behavior-based representation of the downstream LLM. Experiments across three long-term conversational memory datasets and six downstream LLMs show that QARS substantially reduces input tokens while maintaining or improving answer accuracy relative to full retrieved contexts, and outperforms existing compression baselines in accuracy. It further constructs LLM-specific contexts and adapts to unseen downstream LLMs using a calibration set without retraining.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Conversational memory, Context selection, Historical LLM responses

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_X8dqOpenReview ↗
Rating
3
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes QARS, a two-stage post-retrieval context selector for long-term conversational memory. Stage 1 partitions historical LLM responses into positive, uncertain, and discard groups via an incremental semantic-contribution score; Stage 2 trains two binary discriminators (F_keep, F_add) conditioned on a behavior-based LLM fingerprint derived from a diagnostic calibration set. Experiments on ConvoMem, HaluMem, and LongMemEval-S report accuracy gains over RAG and compression baselines with substantially fewer input tokens.

Critical strengths
  1. The paper offers a clear and well-motivated reframing of conversational memory: user queries provide compact semantic anchors, whereas historical LLM responses are treated as conditionally useful evidence rather than uniformly preserved context.
  2. The evaluation spans three datasets, turn- and session-level retrieval, six training-visible and two unseen LLMs, cross-dataset transfer, and targeted ablations, providing considerably broader empirical coverage than a single-benchmark efficiency study.
Critical weaknesses
  1. Although Appendix E.4 varies the upstream retriever, the semantic filtering encoder E(⋅)E(\cdot) before Eq. (2) remains fixed to BGE-M3. Experiments with multiple embedding encoders are needed to establish that QARS generalizes across encoders rather than relying on one relevance geometry.
  2. The paper does not report training or held-out accuracy for FkeepF_{\text{keep}} and FaddF_{\text{add}}, making label learnability and discriminator reliability difficult to assess. Results over multiple random seeds, with variance and validation metrics, are needed to rule out initialization and sampling effects.
  3. Table 1 mixes answer accuracy and token count, so it is difficult to determine whether QARS consistently dominates alternatives or merely selects a different operating point. A Pareto-front plot, ideally with tunable operating points and confidence intervals, would make the accuracy-efficiency comparison substantially clearer.
Questions for authors

See weakness

4 comments
AuthorsOpenReview ↗

Response to Reviewer X8dq - Part (1/2)

Thank you for your thoughtful feedback. We have addressed your concerns in detail below.

Q1: Semantic filtering encoder ablation.

Following your suggestion, we conducted an ablation study on the Stage 1 semantic filtering encoder. Keeping all other QARS components unchanged, we replaced the original BGE-M3 encoder with MPNet and GTE, respectively. The table below reports results averaged over six downstream LLMs.

Filtering encoderConvoMem Acc. (%)ConvoMem Tok.LongMemEval Acc. (%)LongMemEval Tok.
BGE-M370.864382.71430
MPNet68.964781.71394
GTE-large-en-v1.569.466882.01501

Across the three encoders, accuracy varies by at most 1.9% on ConvoMem and 1.0% on LongMemEval, while input token counts differ by at most 25 and 107 tokens, respectively. Although BGE-M3 achieves the highest accuracy, these relatively small variations indicate that QARS maintains stable accuracy and compression efficiency without relying on a specific semantic filtering encoder.

Q2: Discriminator label learnability and seed variance.

We constructed independent evaluation sets for FkeepF_{\text{keep}} and FaddF_{\text{add}} using test data from ConvoMem, HaluMem, and LongMemEval. We repeated discriminator training with two additional random seeds, resulting in three seeds in total, and evaluated both discriminators on these held-out sets. The results are reported as mean ±\pm standard deviation in the table below.

DiscriminatorConvoMem Accuracy (%)HaluMem Accuracy (%)LongMemEval Accuracy (%)
FkeepF_{\text{keep}}71.57 ±\pm 0.4977.80 ±\pm 0.8960.27 ±\pm 0.76
FaddF_{\text{add}}75.67 ±\pm 1.7975.03 ±\pm 1.0577.93 ±\pm 1.76

Both discriminators demonstrate predictive performance on held-out data, with standard deviations ranging from 0.49% to 1.79% across datasets and random seeds. Although accuracy varies across datasets, these results provide evidence of label learnability and relatively stable discriminator performance across random seeds.

AuthorsOpenReview ↗

Response to Reviewer X8dq - Part (2/2)

Q3: Accuracy-token pareto analysis.

We introduced adjustable thresholds for QARS's discriminator output probabilities. For the baseline methods, we varied the compression ratio of LLMLingua2, the number of extracted sentences in RECOMP-Extractive, and the similarity threshold of EXIT to obtain operating points at different token consumption levels. We plotted Accuracy-Token curves on ConvoMem and LongMemEval using Qwen3.5-9B and Qwen3.5-122B as downstream LLMs. The complete curves are shown in Figure 9 in the appendix of the revised PDF.

We constructed empirical Accuracy-Token frontiers from the evaluated operating points without interpolation. At each token level, the frontier represents the highest accuracy among evaluated points using no more than that number of input tokens; the best baseline frontier takes the highest accuracy across all baselines under the same constraint. Over the common token range, we computed five metrics: QARS AUC, Best Baseline AUC, Mean Frontier Gap, Minimum Frontier Gap, and Frontier Superiority Coverage. Normalized AUC measures overall accuracy along the frontier; Mean Gap and Minimum Gap measure QARS's average and minimum accuracy advantages over the best baseline frontier, respectively; and Coverage measures the proportion of the token range where QARS's frontier achieves higher accuracy.

Dataset / LLMQARS AUC ↑\uparrowBest Baseline AUC ↑\uparrowMean Frontier Gap ↑\uparrowMinimum Frontier Gap ↑\uparrowFrontier Superiority Coverage ↑\uparrow
ConvoMem / Qwen3.5-9B68.4763.95+4.52%+2.7%100%
ConvoMem / Qwen3.5-122B70.2867.19+3.09%+1.0%100%
LongMemEval / Qwen3.5-9B82.7878.20+4.58%+4.4%100%
LongMemEval / Qwen3.5-122B81.4678.99+2.47%+1.2%100%

Across all four dataset-LLM settings, QARS achieves higher normalized AUC than the best baseline frontier, with average accuracy gains of 2.47-4.58%. Moreover, the Minimum Frontier Gap remains positive in all settings, and Frontier Superiority Coverage reaches 100% in every setting. This indicates that at different token consumption levels, QARS's empirical frontier consistently lies above the best baseline frontier, demonstrating a sustained accuracy advantage.

Together with the complete curves in the appendix, these findings demonstrate that QARS's advantages are not merely due to selecting different operating points, but reflect a consistently stronger accuracy-efficiency trade-off across token consumption levels.

Thanks author response and revision. I whould like to maintain my overall postitive score, and improve my Informativeness score and Soundness to 3.

AuthorsOpenReview ↗

Thank You for Your Positive Feedback

Thank you for your kind feedback and for acknowledging our responses and revisions. We sincerely appreciate your continued positive assessment of our work and the improved Informativeness and Soundness scores.

Reviewer_pf19OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper observes that in retrieved long-term conversation history, LLM responses make up about 80% of tokens while user queries carry most of the factual content: removing all responses ("Response-Free") costs only 3.9% accuracy on average and cuts 64.1% of tokens. Based on this and the argument that a response's usefulness depen1. Fresh motivation. Most memory and compression methods treat user queries and LLM responses as homogeneous text. Observing that responses are long and largely redundant with the user queries, and treating queries as anchors with responses as optional, is a novel and well-motivated framing that could shape how memory contexts are built.

  1. A trained selector that improves accuracy. Lightweight classifiers trained on LLM-executed labels improve accuracy over full retrieval and compression baselines consistently across six LLMs, both retrieval granularities, an out-of-domain dataset and two unseen LLMs.ds on the query and on the downstream LLM, the authors propose QARS, which always keeps user queries and selects LLM responses in two stages: an embedding-similarity filter, then two small discriminators over frozen embeddings plus a calibration-based LLM representation. The discriminators are trained on labels from running six LLMs on ConvoMem/HaluMem training queries and searching for the minimal sufficient set of responses. QARS improves LLM-judged accuracy over RAG and compression baselines on ConvoMem, HaluMem, LongMemEval and two unseen LLMs, while using fewer tokens than RAG.
Critical strengths
  1. Fresh motivation. Most memory and compression methods treat user queries and LLM responses as homogeneous text. Observing that responses are long and largely redundant with the user queries, and treating queries as anchors with responses as optional, is a novel and well-motivated framing that could shape how memory contexts are built.

  2. A trained selector that improves accuracy. Lightweight classifiers trained on LLM-executed labels improve accuracy over full retrieval and compression baselines consistently across six LLMs, both retrieval granularities, an out-of-domain dataset and two unseen LLMs.

Critical weaknesses
  1. The evidence is benchmark-bound and does not isolate evidence use from parametric knowledge.
  • (a) The premise that responses are "largely compressible" (Response-Free loses 3.9%; responses matter for only 8.9% of queries, Fig. 4) may reflect the benchmarks, where answers are mostly facts stated by the user, and not conversational memory in general. Queries about what the assistant said are common in practice, but no question-type breakdown or assistant-only-evidence test is given, so I am not convinced Response-Free is a genuinely competitive method.
  • (b) It is unclear whether questions are solved by evidence or by parametric knowledge. In Fig. 2 (right), GPT-5.1 answering "Wabi-Sabi" without the response is plausibly guessing from the user's textbook description, not recovering evidence. The same confound enters the training labels (Algorithm 1 marks a response DROP whenever the LLM answers correctly without it) and undermines the "necessity depends on the LLM" claim, since stronger LLMs also guess better. A sounder protocol would first filter for samples that can only be solved from retrieved evidence (e.g., those the LLM fails without memory) and then test compressibility and build the claims on that subset. Without this isolation, the findings cannot be attributed to evidence use.
  • (c) Overall the method gives the impression of being tuned to these benchmarks. QARS alone is trained on in-domain data (4,000 ConvoMem/HaluMem training queries), while all baselines are off-the-shelf and training-free, and the main train/test split is within the same benchmarks. A comparable learned baseline is absent, and LongMemEval is the only out-of-domain test.
  1. The efficiency benefit is not convincing. The headline is saving input tokens, but the reference inputs are small: turn-level RAG uses about 975 (ConvoMem) and 1.7k (HaluMem) tokens, and QARS (about 643 on ConvoMem) saves a few hundred tokens. Response-Free is cheaper than QARS (579), so the savings come mostly from dropping responses.
Questions for authors
  1. Please stratify the main results (RAG, Response-Free, QARS) by whether the LLM answers correctly with no memory. On questions not answerable without memory, how large is the gap between Response-Free and RAG?
  2. Please report results by question type, especially questions whose evidence appears only in assistant turns. How do Response-Free and QARS behave there?
  3. Is there an ablation of the LLM representation on the unseen LLMs (e.g., mean or no representation)? Table 3 only compares with RAG and compression baselines.
5 comments
AuthorsOpenReview ↗

Response to Reviewer pf19 - Part (1/4)

Thank you for your careful review and valuable feedback. We address all your comments below.

Q1: Parametric knowledge confounding.

We would first like to clarify that the response necessity modeled by QARS is relative to a specific LLM and its current query, rather than an absolute measure of the necessity of historical evidence. If an LLM can answer correctly using its parametric knowledge, reasoning capabilities, and remaining context, the corresponding response may no longer be necessary. Differences in LLM knowledge and capabilities therefore lead to different needs for historical responses, which is precisely the motivation behind LLM-aware selection. Likewise, our supervision signals reflect each LLM's task-specific need for a response, rather than an absolute judgment of evidence necessity.

To examine whether QARS's gains depend on questions that LLMs can answer without memory, we have followed your suggestion and stratified the ConvoMem and LongMemEval evaluation samples into Memory-Free Incorrect and Memory-Free Correct, based on each LLM's accuracy without any historical memory. Memory-Free Incorrect accounts for 69.3% of ConvoMem samples and 88.0% of LongMemEval samples. We then report the accuracy and average input token counts of Response-Free, RAG, and QARS on both subsets, as shown in the table below. Values in parentheses indicate accuracy improvements or input token reductions relative to RAG.

ConvoMemMemory-Free Incorrect Acc. (%)Memory-Free Incorrect Tok.Memory-Free Correct Acc. (%)Memory-Free Correct Tok.
Response-Free54.057585.5588
RAG57.897287.5981
QARS (Ours)62.6 (+4.8)649 (-33.2%)94.0 (+6.5)630 (-35.8%)
LongMemEvalMemory-Free Incorrect Acc. (%)Memory-Free Incorrect Tok.Memory-Free Correct Acc. (%)Memory-Free Correct Tok.
Response-Free75.0113388.41086
RAG79.6533791.45366
QARS (Ours)83.3 (+3.7)1434 (-73.1%)92.8 (+1.4)1401 (-73.9%)

On the Memory-Free Incorrect subset, RAG outperforms Response-Free by 3.8% on ConvoMem and 4.6% on LongMemEval. QARS achieves larger gains, outperforming Response-Free and RAG by 8.6% and 4.8% on ConvoMem, and by 8.3% and 3.7% on LongMemEval, respectively, while reducing input tokens by 33.2% and 73.1% relative to RAG. Compared with Response-Free, QARS achieves these accuracy gains with only 74 and 301 additional input tokens per query, respectively, highlighting the value of selectively retained LLM responses.

These results demonstrate that QARS's gains are not driven solely by questions that LLMs can answer without memory. QARS continues to outperform both Response-Free and RAG even on questions that LLMs cannot answer correctly without historical memory.

AuthorsOpenReview ↗

Response to Reviewer pf19 - Part (2/4)

Q2: Assistant-dependent memory evaluation.

We would first like to clarify that the overall performance of Response-Free demonstrates the compression potential of historical responses, while these varying needs further motivate selective response retention. QARS does not assume that responses are generally unimportant; instead, it identifies and retains relevant and useful responses based on the current query and the LLM's capabilities.

Following your suggestion, we have analyzed performance by question type on ConvoMem and LongMemEval, distinguishing questions based on their dependence on historical LLM responses. We define Assistant-Dependent questions as those whose correct answers require evidence provided by historical LLM responses. We classified the questions using Qwen3.8-Max, followed by manual verification by the authors, and labeled the remaining questions as Non-Assistant-Dependent. We report the accuracy and average input tokens of Response-Free, RAG, and QARS, along with Relative Response Token Retention, which estimates the proportion of historical LLM response tokens retained by QARS relative to RAG. The results are shown below.

ConvoMemAssistant-Dependent Acc. (%)Assistant-Dependent Tok.Non-Assistant-Dependent Acc. (%)Non-Assistant-Dependent Tok.
Response-Free52.655267.9589
RAG66.397867.1974
QARS (Ours)73.4 (+7.1)656 (-32.9%)71.9 (+4.8)638 (-34.5%)
Relative Response Token Retention-24.4%-12.7%
LongMemEvalAssistant-Dependent Acc. (%)Assistant-Dependent Tok.Non-Assistant-Dependent Acc. (%)Non-Assistant-Dependent Tok.
Response-Free36.9110083.51133
RAG77.5466381.65458
QARS (Ours)78.4 (+0.9)1887 (-59.5%)85.5 (+3.9)1351 (-75.2%)
Relative Response Token Retention-22.1%-5.0%

On the Assistant-Dependent subset of ConvoMem, QARS outperforms Response-Free and RAG by 20.8% and 7.1%, respectively, while reducing input tokens by 32.9% relative to RAG. On LongMemEval, QARS outperforms Response-Free and RAG by 41.5% and 0.9%, respectively, while reducing input tokens by 59.5% relative to RAG. On the Non-Assistant-Dependent subsets, QARS also outperforms both baselines, achieving accuracy gains of 4.0% and 4.8% over Response-Free and RAG on ConvoMem, and 2.0% and 3.9% on LongMemEval, while reducing input tokens by 34.5% and 75.2%, respectively.

These findings show that historical LLM responses offer substantial compression potential, but this does not mean they can be discarded entirely. By selectively retaining responses that matter for downstream answering based on the current query and the LLM's capabilities, QARS achieves a favorable trade-off between answer accuracy and context compression.

AuthorsOpenReview ↗

Response to Reviewer pf19 - Part (3/4)

Q3: Comparable learned baseline and additional out-of-domain evaluation.

RECOMP-Extractive is a competitive baseline already evaluated in our main experiments. It trains a compressor through contrastive learning to perform sentence-level extraction from retrieved contexts. Following your suggestion, we have adapted the ConvoMem/HaluMem training data used by QARS to the official RECOMP-Extractive training script, trained the baseline, and reevaluated its performance. The table below reports results averaged over six downstream LLMs. As shown in the table, QARS still outperforms the trained RECOMP-Extractive by 6.0% and 7.8% on ConvoMem and LongMemEval, respectively.

ConvoMem Acc. (%)ConvoMem Tok.LongMemEval Acc. (%)LongMemEval Tok.
RECOMP63.974973.61714
trained RECOMP64.875174.91799
QARS (Ours)70.864382.71430

Beyond LongMemEval, we further introduced BEAM [1] (ICLR 2026), which focuses on ultra-long conversational histories, as a second independent out-of-domain dataset. We evaluated QARS on three downstream LLMs: Qwen3.5-9B, Qwen3.5-122B, and Gemma4-26B. As shown in the table below, QARS achieves an average accuracy of 62.5% on BEAM, outperforming Response-Free, RAG, and the trained RECOMP-Extractive by 3.7%, 2.0%, and 2.4%, respectively, while reducing input tokens by 70.5% relative to RAG.

BEAMQwen3.5-9B Acc. (%)Qwen3.5-9B Tok.Qwen3.5-122B Acc. (%)Qwen3.5-122B Tok.Gemma4-26B Acc. (%)Gemma4-26B Tok.Avg Acc. (%)Avg Tok.
Response-Free57.9151259.7151258.7151258.81512
RAG60.01172162.01172159.51172160.511721
trained RECOMP59.9380561.1380559.2380560.13805
QARS (Ours)62.1354363.3397062.2284462.53452

[1] Tavakoli, Mohammad, et al. "Beyond a million tokens: Benchmarking and enhancing long-term memory in LLMs." International Conference on Learning Representations (ICLR). 2026.

Reviewer_XuvzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies retrieval-based long-term conversational memory and asks whether the assistant responses in retrieved history turns are actually needed. The authors find that responses take up about 80% of the tokens but can mostly be dropped. They propose QARS, which keeps all user queries, filters responses by embedding similarity, and then uses two small classifiers to decide which responses to keep. The classifiers are conditioned on how the downstream model answers a fixed calibration set, so new models can be added without retraining. Across three datasets and eight downstream models, QARS is more accurate than plain RAG with fewer input tokens, and more accurate than existing post-retrieval compression methods.

Critical strengths
  1. Separating user queries from assistant responses seems like the right angle to me. Most prior memory work stores and retrieves whole turns. In Table 1, the Oracle uses only a few dozen more tokens than Response-Free but is close to 20 points more accurate, which makes the point quite clearly.

  2. Representing the downstream model by its answers on a calibration set is a neat idea, and a new model only needs one pass over that set. The swap experiments in Appendix G, where using another model’s representation hurts accuracy, at least show that different models do need different responses.

  3. The evaluation is fairly thorough. LongMemEval is held out, GPT-5.1 and Llama are unseen, and the authors also vary the retriever and the retrieval granularity. As a post-retrieval plug-in, 0.45s per query is acceptable for a real system.

Critical weaknesses
  1. The evaluation only checks whether the final answer is correct, but the method is designed to drop responses the model can “recover on its own,” so there is a real risk of rewarding world knowledge in place of memory. In Case 1, GLM explicitly says the relevant response is not in the history and then answers Veja from general knowledge, which counts as correct under your metric even though nothing was recalled. I would like to see results on the LongMemEval questions that ask about what the assistant said, or on cases where the assistant’s earlier answer disagrees with common knowledge.

  2. With only six training models, the discriminators see only six distinct eLe_L vectors, so I am not convinced they learn model-specific selection in a way that generalizes. Could you compare the unseen-model results in Table 3 against simply using the mean of the six training representations? If the gap is small, the contribution of the LLM-aware part is weaker than claimed.

  3. A simple baseline is missing: keep all user queries and add back only the top-1 or top-2 responses by similarity. The compression baselines are trained on other data and also compress the user queries, so it is hard to tell whether the gains come from the two-stage selector or mostly from leaving the user queries untouched.

Questions for authors

see weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer Xuvz - Part (1/2)

Thank you for your thoughtful feedback. We have addressed each of your concerns as follows.

Q1: Performance on assistant-dependent questions in LongMemEval.

We would first like to clarify the objective of QARS: to select historical LLM responses needed by a specific LLM to answer the current query, rather than to require answers to rely exclusively on historical conversational evidence. LLMs differ in their parametric knowledge and reasoning capabilities, and thus may have different needs for the same historical response. If an LLM can answer correctly without a particular response, removing it is reasonable from a task-oriented memory compression perspective. In Case 1, GLM can correctly answer "Veja" without the corresponding historical response; retaining that response would therefore increase input tokens without improving answer accuracy for GLM. This illustrates the central design principle of QARS: response necessity is determined by its contribution to a particular LLM's downstream answering performance, rather than solely by whether it contains the explicit textual evidence.

However, this does not imply that all historical LLM responses can be replaced by parametric knowledge. Following your suggestion, we have conducted an assistant-dependence analysis on LongMemEval. Specifically, we define Assistant-Dependent questions as those whose correct answers require evidence provided by historical LLM responses. These primarily include questions about what the assistant previously said or recommended. We classified the questions using Qwen3.8-Max, followed by manual verification by the authors. The remaining questions were classified as Non-Assistant-Dependent.

Assistant-Dependent Acc. (%)Assistant-Dependent Tok.Non-Assistant-Dependent Acc. (%)Non-Assistant-Dependent Tok.
Response-Free36.9110083.51133
RAG77.5466381.65458
QARS (Ours)78.4 (+0.9)1887 (-59.5%)85.5 (+3.9)1351 (-75.2%)
Relative Response Token Retention-22.1%-5.0%

As shown in the table, on the Assistant-Dependent subset, the 40.6% accuracy gap between Response-Free and RAG highlights the importance of historical assistant responses. QARS outperforms Response-Free and RAG by 41.5% and 0.9%, respectively, while reducing input tokens by 59.5% relative to RAG. On the Non-Assistant-Dependent subset, QARS outperforms Response-Free and RAG by 2.0% and 3.9%, respectively, while reducing input tokens by 75.2% relative to RAG. We additionally report Relative Response Token Retention, which approximates the proportion of historical LLM response tokens retained by QARS relative to RAG. This ratio is 22.1% for Assistant-Dependent questions, substantially higher than the 5.0% for Non-Assistant-Dependent questions, indicating that QARS retains relatively more historical LLM response content when assistant-provided information is relevant, while achieving substantial input-token savings and maintaining strong answer accuracy. We also include a new case study in the appendix of the revised PDF, illustrating how QARS preserves critical historical responses.

AuthorsOpenReview ↗

Response to Reviewer Xuvz - Part (2/2)

Q2: Mean representation ablation on unseen LLMs.

Following your suggestion, we have evaluated the mean representation of the six training LLMs (Mean) on two unseen LLMs, Llama3.3-70B and GPT-5.1, and further conducted an ablation without LLM representations (No). The results are shown in the two tables below.

Llama3.3-70BConvoMem Acc. (%)ConvoMem Tok.LongMemEval Acc. (%)LongMemEval Tok.
QARS (Ours)67.463780.41321
Mean Representation63.8 (-3.6)62875.6 (-4.8)1256
No Representation62.7 (-4.7)61874.0 (-6.4)1207
GPT-5.1ConvoMem Acc. (%)ConvoMem Tok.LongMemEval Acc. (%)LongMemEval Tok.
QARS (Ours)79.360386.61243
Mean Representation76.1 (-3.2)62882.8 (-3.8)1256
No Representation75.5 (-3.8)61881.2 (-5.4)1207

As shown in the two tables, QARS with each target LLM's own behavioral representation (Ours) consistently outperforms both Mean and No on Llama3.3-70B and GPT-5.1 across ConvoMem and LongMemEval. Replacing Ours with Mean reduces average accuracy by 3.4% and 4.3% on ConvoMem and LongMemEval, respectively, while removing LLM representations leads to drops of 4.3% and 5.9%. These results indicate that each LLM's behavioral representation captures LLM-specific information that cannot be replaced by the mean training LLM representation, improving response selection on unseen LLMs and supporting the cross-LLM generalizability of the LLM-aware component.

Q3: Baselines retaining all historical user queries.

Following your suggestion, we have added baselines that retain all user queries and select the top-1 or top-2 LLM responses based on semantic similarity. The table below reports results averaged over six downstream LLMs.

ConvoMem Acc. (%)ConvoMem Tok.LongMemEval Acc. (%)LongMemEval Tok.
QARS (Ours)70.864382.71430
Top-1 Response67.5 (-3.3)622 (-21)80.1 (-2.6)1547 (+117)
Top-2 Response68.1 (-2.7)661 (+18)79.7 (-3.0)1964 (+534)

On ConvoMem, QARS outperforms Top-1 and Top-2 by 3.3% and 2.7%, respectively. On LongMemEval, QARS outperforms Top-1 and Top-2 by 2.6% and 3.0%, respectively, while reducing input tokens by 7.6% and 27.2%. These results demonstrate that QARS's gains are not solely attributable to preserving all user queries; its two-stage, LLM-aware response selection also provides additional benefits.

Thank you for the response, which addresses some of my concerns. However, given the limited novelty, I will keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to Reviewers’ Comments and Summary of Revisions

Dear Area Chairs and Reviewers,

We sincerely thank you for your careful evaluation and constructive feedback. We have provided detailed responses to the reviewers' comments and updated the manuscript accordingly. Below, we summarize the strengths recognized by the reviewers, our responses to the main concerns, and the key revisions made to the manuscript.


Strengths Recognized by Reviewers

We sincerely thank the reviewers for recognizing the following strengths of our work:

  • Novel and well-motivated research perspective (Xuvz, pf19, X8dq). All three reviewers positively recognized the distinction between historical user queries and LLM responses. Reviewer Xuvz noted that this "seems like the right angle," reviewer pf19 described it as "a novel and well-motivated framing," and reviewer X8dq characterized it as "a clear and well-motivated reframing."

  • Effective LLM-aware response selection design (Xuvz, pf19). Reviewer Xuvz described the calibration-based LLM representation as "a neat idea," while reviewer pf19 highlighted the learned response selector as a strength, noting its effectiveness in improving accuracy.

  • Comprehensive empirical evaluation (Xuvz, pf19, X8dq). All three reviewers recognized the extensive evaluation across multiple datasets, retrieval granularities, and downstream LLMs, as well as the demonstrated improvements in accuracy and token efficiency.


Main Concerns and Our Responses

  • Historical evidence utilization and parametric knowledge (Xuvz, pf19): Additional analyses of assistant dependence and memory-free correctness demonstrate that QARS effectively preserves useful historical evidence while improving accuracy and reducing input tokens.

  • Effectiveness of the LLM-aware mechanism and robustness (Xuvz, pf19, X8dq): Ablations of behavioral representations on unseen LLMs, alternative semantic encoders, and random seeds validate the effectiveness and stability of QARS.

  • Baseline comparisons and generalization (Xuvz, pf19): Additional comparisons with Top-1/Top-2 response selection and trained RECOMP-Extractive, together with evaluations on an additional out-of-domain dataset, BEAM, further demonstrate QARS's accuracy and token-efficiency advantages.

  • Accuracy–token efficiency trade-off (pf19, X8dq): Pareto frontier analyses demonstrate consistent accuracy-efficiency advantages across different levels of token consumption.


Summary of Manuscript Revisions

The revised manuscript includes the following key updates:

  • Sections 5.5 and 5.6: Added analyses stratified by assistant dependence and memory-free correctness to further examine the retention of historical assistant-provided information and the influence of parametric knowledge.

  • Appendices E.1 and E.2: Added Top-1/Top-2 response selection baselines, a trained RECOMP-Extractive comparison, and additional out-of-domain generalization experiments on BEAM.

  • Appendices F.5, F.6, and H.2: Added semantic filtering encoder ablations, discriminator stability analysis across multiple random seeds, and behavioral representation ablations on unseen LLMs.

  • Appendix I (Figure 9): Added Accuracy–Token curves and Pareto frontier analyses across different levels of token consumption.

  • Appendix K (Case 3): Added a case study requiring historical assistant-provided information to illustrate how QARS preserves critical evidence for downstream answering.


We sincerely appreciate your thoughtful feedback and hope that our responses and revisions address your concerns and contribute to a favorable reassessment of our work.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.