- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies information reuse and detail preservation in long-term conversational memory. Because the importance of a fact depends on subsequent questions, premature compression may omit necessary details, whereas processing the full raw history for each question incurs substantial repeated overhead. The authors propose GiDe, a training-free two-stage method. First, it extracts backbone facts with source information from the dialogue history and repairs them through a QA-based checking procedure. Then, given the current question, it selects backbone evidence, identifies information gaps, and, when necessary, consults the original dialogue to construct the final task memory. The paper analyzes the method’s applicable conditions using a simplified information-theoretic model, evaluates question-answering performance on LoCoMo and LongMemEval-S, and supplements the study with component ablation, controlled retrieval, cross-model transfer, and amortized token-cost experiments.
Critical strengths
- The method addresses a clear and practically meaningful tension, and the division of labor between the two stages is well justified. When future questions are unknown, compressed memory struggles to retain all potentially necessary details. GiDe combines reusable facts with task-time detail recovery, allowing the backbone to both provide partial answers and help locate original evidence. In Table 3, removing Stage 1 causes a 3.07-point drop in F1, and removing detail recovery together with task-memory assembly causes a 6.97-point drop, supporting the contribution of both stages to the final performance. This gives the paper’s design a relatively clear empirical basis.
- The experiments support the practical value of the method from multiple angles. Under the GPT-4o-mini configuration, GiDe achieves an overall F1 of 59.76 on LoCoMo and an overall score of 84.0 on LongMemEval-S, with particularly strong performance on multi-hop and temporal reasoning categories. Controlled retrieval experiments further show that backbone clues can help locate necessary facts amid substantial distractor information; cross-model experiments and amortized cost experiments also provide evidence for the benefits of reuse. Taken together, these results suggest that the framework is worth further research and adoption.
Critical weaknesses
- Lack of direct comparison with the closest methods. Section 2.2 notes that ReadAgent, MemWalker, and AnchorMem already adopt a “pre-built summary or factual线索, then look up the original text” approach, but these methods are not included in the main experiments. As a result, the current results are insufficient to clearly establish GiDe’s additional contribution relative to similar frameworks. It is recommended to include at least one closest method and compare answer quality and processing cost under a consistent evaluation setup.
- The triggering condition for looking back at the original dialogue is unclear, and the backbone-first retrieval order lacks validation. Section 3.3 describes looking back at the original text when an information gap remains unresolved, while Appendices A.3 and A.10 link the look-back step to exhaustion of the backbone candidate pool and cap the retrieval loop at three steps. It is unclear how the system behaves when a gap persists, the candidate pool is not yet exhausted, but the step limit has been reached. Moreover, the paper does not compare a strategy that looks back at the original dialogue immediately after detecting a gap, making it difficult to judge the actual value of continuing to search the backbone in terms of quality and efficiency.
- The effectiveness of incremental updating is not supported by experiments. Section 3.2 and Appendix A.2 describe local updates when a new session arrives, but the experiments do not show answer quality or update cost as sessions are continuously added. In particular, when new information revises existing preferences, plans, or event states, it remains unclear whether local updates can maintain temporal consistency in memory. This limits the evidence for backbone reuse in sustained interaction scenarios.
- The handling of the EXTERNAL branch is unclear, affecting the interpretation of open-domain results. Section 3.1 assumes that answers are supported by the dialogue history, whereas Appendix A.3 allows returning EXTERNAL for general-knowledge questions, even though the reader is required to answer solely based on the provided evidence. The paper does not specify what happens after entering this branch, how often it is triggered, or whether comparison methods follow the same knowledge-use rules. Therefore, it is unclear how much of the open-domain gain can be attributed to memory selection and detail recovery.
- The reliability of subsequent recovery mechanisms when the initial backbone localization is wrong is not analyzed. GiDe uses person or event clues from the selected backbone to construct subsequent queries. If the initial retrieval links to a similar but incorrect event, supplementary retrieval may continue following the wrong clues to seek details. Section 4.2 considers clue misidentification, but the experiments do not show recovery performance when such errors actually occur. Thus, whether the method can correct initial mislocalization, and how strongly its gains depend on localization accuracy, requires further clarification.
Questions for authors
- Are there any direct comparisons with ReadAgent, MemWalker, or AnchorMem?
- When the knowledge gap persists but the retrieval step limit is reached, does the system necessarily fall back to the original text? Compared with immediate fallback, what are the empirical benefits of the backbone-first strategy?
- Has the answer quality and update overhead been evaluated when new sessions are incrementally added?
- How exactly does the EXTERNAL branch respond, how many times was it triggered, and are the knowledge usage rules consistent across all compared methods?
- If the initial localization of a person or event is incorrect, can the system correct it? Are there any relevant cases or statistics on the recovery success rate?
5 comments
Response to Reviewer Ae3s (2/3)
[2/3] W3–W4, Q3–Q4. Incremental updates and EXTERNAL
Adding new history (W3, Q3). We evaluated addition and revision separately, since retaining new facts and reconciling changed facts pose different requirements. For addition, we kept the existing conv-26 backbone and processed conv-30’s 19 raw sessions through extraction, probing and repair before merging the new facts. This update costs 148,326 tokens across 156 calls. On the same 233 questions, the combined bank scores 56.65 F1 versus 56.78 for separately constructed and queried banks; online costs are 7,133 versus 7,243 tokens per question. This evaluates retrieval from the combined history after updating the existing bank (Section 5.5, Appendix B.4 and Table 10).
Table 10. Trajectory-addition results (revised manuscript, Appendix B.4)
| Method | F1 ↑ | Accuracy (%) ↑ | Online tokens/question ↓ |
|---|---|---|---|
| Incrementally update | 56.65 | 87.55 | 7,133 |
| Separate construction and retrieval | 56.78 | 88.41 | 7,243 |
Revising current and historical states (W3, Q3). To examine the temporal-consistency concern, we added one synthetic revision session to conv-41. Editing the existing backbone and rebuilding from the revised history are evaluated on the same 16 questions: eight ask about current states and eight about historical states. The historical questions check whether earlier information remains available after a change. Both answering procedures and backbone editing use GPT-4o-mini; accuracy is automatically judged (Section 5.5, Appendix B.4 and Table 11).
Table 11. State-revision pilot (revised manuscript, Appendix B.4)
| Update policy | Update-stage tokens ↓ | F1 ↑ | Automatically judged ACC ↑ |
|---|---|---|---|
| Edit existing backbone | 17,932 | 80.45 | 100.0% |
| Rebuild backbone | 257,492 | 71.37 | 87.5% |
Update quality and cost. Editing uses 17,932 update tokens versus 257,492 for rebuilding, a 93.0% reduction, with F1 of 80.45 versus 71.37 and model-judged correctness of 16/16 versus 14/16. Answering all 16 questions correctly under this judge covers both the current-state and historical-state questions in the controlled revision. Together, Tables 10–11 provide quality and cost measurements for one trajectory addition and one revision session; repeated session-by-session updates over a long interaction stream were not evaluated (Appendix B.4).
EXTERNAL execution (W4, Q4). The EXTERNAL label in the controller prompt does not open a separate general-knowledge answering branch. In the evaluated path it is mapped to MISSING: GiDe continues retrieving from the conversation, and the final reader answers using the supplied context. Thus, EXTERNAL follows the same dialogue-retrieval procedure as other insufficient-context decisions, consistent with Section 3.1’s history-supported setting (Appendix A.3). Raw pre-mapping EXTERNAL frequencies were not retained. Baselines follow their native prompts, so knowledge-use rules were not separately standardized across methods (Section 5.1).
Interpreting open-domain performance. Appendix A.3 includes a separate question-only diagnostic on the 96 open-domain questions: 9 strictly correct answers, 4 incorrect answers and 83 exact abstentions. This shows that the answer model can supply some answers without retrieved dialogue. These counts describe question-only behavior; they do not decompose the model-knowledge and retrieval contributions to GiDe’s final score. We report the open-domain comparison as answer performance of the complete methods under the settings above (Tables 2–3).
Response to Reviewer Ae3s (3/3)
[3/3] W5, Q5. Recovery from incorrect localization
Correction after an initial localization error (W5, Q5). We examined 385 questions from conv-26/30/41 using automatic post-hoc annotations of the retrieved context. Fifteen initially target the wrong person or event (3.9%); five have missing or confusing backbone clues (1.3%), overlapping the first group. Among 13 initially mislocalized questions with a first-round MISSING verdict and comparable later annotations, ten are labeled correctly localized after retrieval and recovery (S3), or 76.9%. These are context-localization labels for a defined subset, distinct from final answer accuracy (Appendix B.3 and Table 9).
Table 9. Localization diagnostics (selected rows; revised manuscript, Appendix B.3)
| Diagnostic | Count | Meaning |
|---|---|---|
| Wrong initial person/event localization | 15 / 385 | Automatically annotated initial retrieval error |
| Missing or confusing backbone clue | 5 / 385 | Overlaps with the first category |
| Corrected at S3 in the defined MISSING subset | 10 / 13 (76.9%) | Conditional pipeline localization correction |
How the search target can change (W5, Q5). At later steps, the controller generates a focused query from the original question and the facts selected so far; source lookup uses the latest query, or the original question if none was generated. An initial person or event clue can therefore be revised as the context changes. The 10/13 result shows that initial errors can be corrected during this process. Three of these 13 cases were not labeled correctly localized at S3, so correction is not automatic. The statistic covers the subsequent retrieval-and-recovery process; it does not isolate source lookup alone or quantify final F1’s dependence on initial localization accuracy (Section 3.3 and Appendix B.3).
Complementary retrieval results. The 783-question test in Figure 5. Retrieval recall under distractors (anonymous link) examines source retrieval under distractors. Joint backbone-fact and source-session guidance raises top-3 recall of annotated dialogue turns from 61.94% to 71.26% at 99% target distractors. The same questions are used across distractor ratios. A separate fact-only test at 90% distractors changes recall by at most 0.005 across reported budgets (Section 5.5 and Appendix B.2). These tests measure retrieval guidance in their respective settings; Table 9 provides the direct observations of initial localization errors and subsequent correction. Evaluation question list (anonymous link).
Response to Reviewer Ae3s (1/3)
We thank the reviewer for the careful assessment and constructive suggestions. Section, table, and figure numbers below refer to the revised manuscript.
[1/3] W1–W2, Q1–Q2. Closest methods and source lookup
Direct comparisons (W1, Q1). We added ReadAgent and AnchorMem on the same ten LoCoMo conversations and 1,540 non-adversarial questions. Within each model group, methods share the answer model and token-level F1 scorer, retaining their native prompts, retrieval and final-context policies. Table 5 below reports both answer quality and total construction-plus-answering tokens. This directly compares two of the closest source-lookup methods under the reported evaluation protocol (Sections 5.1–5.2 and 5.4; LoCoMo category scores in Tables 2–3).
Table 5. LoCoMo quality–cost comparison (selected rows; revised manuscript, Section 5.4)
What GiDe adds (W1, Q1). With GPT-4o-mini, GiDe improves F1 over ReadAgent and AnchorMem by 28.65 and 11.58 points while reducing total tokens by 5.5% and 15.6%. The quality ordering also holds with Qwen2.5-14B: 57.59 for GiDe, 36.91 for ReadAgent and 39.41 for AnchorMem. ReadAgent uses page summaries, and AnchorMem uses fact anchors and associated events. GiDe couples QA repair of compact facts with gap-directed source queries, so the same repaired facts support answering and help find omitted details (Table 1). The component results clarify this contribution: removing QA repair lowers F1 by 2.61 points; removing detail recovery lowers it by 9.73; using the question alone for source lookup lowers it by 3.22 (Section 5.3, Table 4). The quality–cost frontier is shown in Figure 3.
Source-lookup trigger (W2, Q2). Yes: if a gap persists at the three-step limit, source lookup is triggered even when unused backbone candidates remain. GiDe searches the backbone for at most three steps, followed by a final sufficiency check; candidate exhaustion can also trigger lookup. When the selected facts suffice, regular source lookup is skipped, while reader refusal can independently trigger recovery. This rule lets another stored fact fill a gap before consulting the original dialogue, while bounding the backbone search (Section 3.3 and Appendix A.3).
What the lookup ablations establish (W2, Q2). Selective recovery reaches 59.81 F1 at 7,128 online tokens per question, versus 57.60 at 10,489 for unconditional lookup after backbone search. Both variants first search the backbone; the unconditional variant also retrieves raw passages when the context is already judged sufficient. Thus, selective supplementation improves quality and reduces online cost in this comparison. GiDe also answers 386/1,540 questions without source lookup, with 85.23% model-judged accuracy on that subset, illustrating the backbone’s direct-answering role (Section 5.5). The specific comparison with immediate lookup at the first detected gap was not evaluated; Table 4 tests selective versus unconditional supplementation after search (Section 5.3).
Table 4. Source-lookup ablation (selected rows; revised manuscript, Section 5.3)