- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of missing retrieval intent in long-running conversations: users’ follow-up requests often omit project, entity, configuration, or version information established earlier, making it difficult for conventional RAG systems to retrieve the correct evidence from the current query alone. The paper proposes Parametric Session State (PSS), which compiles conversational information into a session-specific LoRA state through history language modeling and supervised readout of positive and negative retrieval cues. At inference time, this state is temporarily loaded to generate retrieval cues, which are passed to a frozen retriever to query external sources. The state is then unloaded, and the frozen base model produces an answer grounded in the retrieved evidence.
The method is evaluated on the synthetic PSS-320 dataset, conversational adaptations of public QA tasks, UltraDomain, and multiple base models and retrieval backends. Wrong-session, shuffled-state, and target-only controls are used to investigate the sources of improvement. The results support the effectiveness of PSS in improving evidence retrieval under the specified supervision conditions and indicate that conversational history contributes information beyond the explicitly provided retrieval targets.
Critical strengths
S1. Targeted methodological design. The method uses session state to recover missing retrieval information and decouples retrieval from answer generation, ensuring that the final answer remains grounded in inspectable external evidence.
S2. Substantial controlled experimentation. Comparisons involving incorrect states, shuffled states, and different training inputs examine whether the gains arise from information specific to the corresponding session, strengthening the credibility of the findings.
S3. Broad empirical coverage. The evaluation spans multiple datasets, language models, and retrieval backends. It also demonstrates that retrieval improvements can benefit downstream answering, providing evidence of applicability across different settings.
Critical weaknesses
W1. The practical task formulation and motivation for the method are insufficiently convincing.
Retrieval failures caused by underspecified follow-up queries are a legitimate problem. However, the paper does not sufficiently establish how its task formulation corresponds to the practical needs of long-running assistants. In the example shown in Figure 1, a single clarification exchange with the user appears sufficient to resolve the ambiguity.
The proposed offline training procedure requires some of the key information linking the query to its session context. Consequently, the evaluation primarily examines how to store and recover an already established retrieval scope, rather than fully addressing intent recognition in natural conversation. Under this assumption, directly storing the parsed information as text is a natural and efficient alternative. Why, then, is it necessary to encode this information in trained parameters? In Figure 3, Parsed reference achieves a retrieval score of 96.95%, compared with 96.56% for PSS. Substantial improvements over the information-deficient Raw query baseline therefore do not, by themselves, establish the need for session-specific LoRA training.
Although the introduction motivates the work through long-running assistants, the main paper primarily demonstrates readout from a fixed conversational history after it has been encoded. It does not clearly explain how the state is updated as the conversation evolves, how long a trained state remains reusable, or how the training and maintenance costs are amortized. The paper has not demonstrated that the benefits justify the additional parameter training and maintenance overhead introduced by PSS.
W2. The method primarily combines existing components, and its technical novelty is insufficiently established.
The method combines existing LoRA adaptation, history language modeling, positive and negative cue supervision, and external retrieval. Its main distinction is maintaining a parameter state for each session and loading it only before retrieval. This workflow has some design merit, but the paper does not sufficiently clarify what distinctive capability or learning mechanism it introduces beyond existing approaches to parametric memory, memory-assisted cue generation, and conversational query rewriting.
In particular, the authors identify MemoRAG as the closest work in terms of functional decomposition. A clearer explanation is therefore needed as to whether PSS differs primarily in where information is stored and how components are invoked, or whether these differences enable a new capability. The existing mechanism controls can establish that the correct state contains useful session information, but they cannot independently establish that storing this information in parameters is preferable to storing it as text.
W3. The experimental comparisons with closely related baselines are insufficient.
The paper provides a substantial set of internal baselines and mechanism controls, but direct comparisons with the closest external methods remain limited. For example, MemoRAG is explicitly identified as the closest approach in terms of functional decomposition, yet no corresponding experimental results are reported. Related parametric memory methods are also discussed primarily at a conceptual level.
The external conversational query rewriting comparison in the main paper largely relies on ConvSearch-R1. Meanwhile, experiments across retrieval backends primarily establish interface compatibility and cannot substitute for competitive comparisons with complete memory-based systems. The paper should provide same-task comparisons with representative closely related methods, including relevant approaches discussed in Related Work, or explain concretely why a fair comparison is not feasible.
W4. The organization and presentation lack clarity, making the central contributions difficult to identify and understand efficiently.
- Figure 2 lacks an explicit reference and a step-by-step explanation in the main text. Although the figure has a caption, the method description does not walk the reader through its inputs, training stages, and online workflow.
- The method description emphasizes formal definitions without adequately explaining the core contribution. Although the losses are specified, the paper lacks a complete worked example showing the correspondence among conversational history, positive and negative cues, training prompts, and the online query.
- The experimental configurations are not explained consistently. For settings such as Card, Summary, Parsed reference, Gold query, and Guarded PSS, the information sources, budgets, and experimental roles are not presented in a unified manner, making their meanings and differences difficult to understand.
- The presentation switches frequently among datasets and protocols. The original PSS-320 dataset, public conversational datasets, UltraDomain, newly introduced carrier datasets, and multi-record experiments are interleaved. The two similarly named datasets containing 320 sessions are particularly easy to confuse.
- The research questions lack a clear organizing thread. RQ1 covers retrieval, question answering, domain transfer, retrieval-backend transfer, and online workload, while RQ3 covers hyperparameters, the contribution of history, carrier selection, and multi-record access. Moreover, nearly every paragraph in the experimental section provides only a brief statement followed by an appendix reference, making it difficult to obtain sufficient substantive information from the main text.
- The main paper is not sufficiently self-contained, and appendix references are fragmented. For example, §3.2 directs readers to Appendices B, I, J, L, M, and S for dataset, baseline, prompt, and statistical details, while information about the method and its costs is further distributed across Appendices J, R, and V.5.
Questions for authors
Q1. What is the concrete application scenario? Asking the user for clarification is a natural solution to the ambiguity illustrated in Figure 1. In which practical scenarios is such interaction infeasible or sufficiently costly to motivate automatic intent recovery through PSS?
Q2. What concrete benefit does parametric storage offer over textual storage? In Figure 3, Parsed reference achieves 96.95%, slightly exceeding PSS at 96.56%. Given that retrieval cues are already available during training, under what conditions is training a LoRA state more valuable than directly storing and using those cues as text?
Q3. How is the state updated as the conversation evolves, and how is cost efficiency maintained? When users introduce new information, revise their goals, or revoke previously specified configurations, how does PSS update its state? Could the authors provide a cost analysis that includes repeated updates and identifies the level of reuse required to obtain a practical benefit?
Q4. What is the core technical innovation relative to existing memory methods? Compared with MemoRAG, parametric memory, and conversational query rewriting approaches, what distinctive capability does PSS introduce beyond changes to the storage format and invocation workflow? Which experiments directly validate that capability?
Q5. Can the authors provide fair comparisons with the closest related methods? Experiments across retrieval backends primarily demonstrate compatibility. Could MemoRAG or other representative memory methods be evaluated under the same task, available information, and resource budget? If such comparisons are infeasible, please explain the specific reasons.
Q6. Can the main paper include the essential information needed to understand the method and experiments? A complete worked example linking conversational history, positive and negative cues, training, and online querying would be helpful. The paper should also consolidate the information sources and budgets of the baselines and clarify the relationships among datasets, protocols, and research questions, reducing dependence on scattered appendix sections.