acceptodds
Under review as a conference paper at ICLR 2027

From Records to Beliefs: NOBLE for Evidence-Grounded Personalized Memory

Abstract

Long-term personalized agents must infer user states from historical interactions to complete subsequent tasks, yet these states are only partially observable and non-stationary. This requires memory systems to continually revise their understanding of the user, a capability that remains insufficiently supported by methods focused on compressing, organizing, and retrieving historical information. We present NOBLE, an evidence-driven memory system that shifts from managing historical records to maintaining revisable beliefs. NOBLE treats interactions as observational evidence for constructing and updating hypotheses about user states; these hypotheses carry confidence estimates and collectively form the system's evolving belief state. Dynamic themes organize the hypotheses and their associated evidence across emerging user dimensions, guiding both theme-local updates and task-relevant retrieval. This organization enables NOBLE to update its understanding as new evidence arrives while preserving its evidential basis, and to assemble compact context for downstream tasks. We evaluate NOBLE on VitaBench 2.0 for long-term personalized task execution and it achieves 41.0% Avg@4 with GPT-4.1, outperforming all evaluated memory baselines. Evaluation on LongMemEval also demonstrates its ability to support long-term conversational question answering.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
agent, memory, long horizon task, personalized memory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_u1T8OpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes NOBLE\texttt{NOBLE} as an external memory framework for long-term personalization via LLM agents. NOBLE\texttt{NOBLE} maintains an explicit, revisable hypothesis that is grounded to the evidence in the history. It first converts each interaction into compact evidence, uses it to create an evolving hypothesis of the user, revises the hypothesis (updates the current belief state) based on newer information, and finally performs a personalized action after retrieval. NOBLE\texttt{NOBLE} does not train a new personalization model from the user sequences; rather, it uses PLLMs to interpret interactions and an explicit memory structure, which helps to store, revise, retrieve, and ignore the beliefs about the user. The central motivation stated is that raw observations of the user's actions do not provide the true underlying interests of the user. Therefore, this interest state remains partially observable by the LLM agent. Results show that NOBLE\texttt{NOBLE} completes more tasks successfully that compared baselines.

Critical strengths

1: Explicit decomposition of memory into evidence-gorunded, revisable beliefs for personalized tasks

The main strength I see is the explicit decomposition framed for a personalization process. The separation of raw interaction history from its interpretations: interactions being converted into atomic traceable evidence, and then being organized into dynamic themes. The explicit hypothesis containing proposition, applicability conditions, confidence, supporting and conflicting evidences, is a genuine interesting idea, although not completely new. Transparent, Scrutable and Explainable User Models for Personalized Recommendation; SIGIR 2019\textit{Transparent, Scrutable and Explainable User Models for Personalized Recommendation; SIGIR 2019}, Belief Revision: The Adaptability of LLM Reasoning; EMNLP 2024\textit{Belief Revision: The Adaptability of LLM Reasoning; EMNLP 2024} already explores the areas of explicit natural-language user preferences, conditional preferences/exceptions, interpretable/editable user model, and concept of belief revision in long-sequence LLM tasks. I still consider this to strength provided that NOBLE\texttt{NOBLE} is positioned around them more clearly, since the core formulation of evidence provenance, positive/negative evidence sets, confidence dynamics remain distinct.

2: The structured updating of new evidences

The framework interprets new evidences relative to the existing user hypotheses and performs multiple update actions SUPPORT, REFINE, SUPERSEDE\textbf{SUPPORT, REFINE, SUPERSEDE} and so on, which allows the agents to understand and distinguish repetation, obsolete relation, conditional refinement and one-off noise in user interaction history.

Critical weaknesses

1: Problem Framing has scope issue

The problem statement is framed around partially observable user states, non-stationarity of preference evolution, and need for continual revision of user hypotheses. However, in sequential personalization, fundamental idea that user state depends on previous history and the newest interaction, already exists. Recurrent models, transformer-based models such as Mamba4rec, GRU4Rec, LSTUR, NAML, NRMS, already update their internal representation of user as new evidences arrive. More specialized models also explicitly distinguish different temporal components of user interests which are long-term, short-term as well as episodic bursty interests. Hence the novelty seems to be externalization of user state into evidence-grounded hypotheses instead of \textbf{\textit{user understanding should revise the model's belief as new evidences arrive}}.

2: Limited experimental coverage relative to personalization

Vitabench2 is used as evaluation for end-to-end personalized task execution. It evaluates the full downstream action pipeline, and the final success rate establishes the downstream utility of the full memory system, but does not measure the quality of intermediate belief representation itself. However, the question remains to be investigated is whether introduction of this explicit evidence-hypothesis-belief structure improve next timestep predictive tasks\textit{whether introduction of this explicit evidence-hypothesis-belief structure improve next timestep predictive tasks}

such as sequential recommendation, or personalized content generation, relative to strong sequential personalization models\textit{such as sequential recommendation, or personalized content generation, relative to strong sequential personalization models}. Another related concern is whether SOTA sequential recommenders perform better than their current performances when explicit evidences are decomposed and then fed to their per-timestep user latent represnetations.

Questions for authors
  1. Is it impossible to apply NOBLE\texttt{NOBLE} to personalization benchmarks such as MIND news, PENS headline personalization, MovieLens movie prediction, and Amazon product recommendation tasks?

  2. Is there a learning of the hypotheses update? Can you provide whether the hypotheses actually provides quality revisions?

  3. Can there be standard metric-based evaluations such as NDCG, MRR, HitK?

2 comments

We thank the reviewer for the constructive comments on NOBLE's positioning and evaluation.

W1

We agree that sequential personalization already models evolving user interests, including long- and short-term components. NOBLE's contribution lies in operationalizing evidence-grounded belief revision as an external memory mechanism for personalized agents. Its update unit is an individual interpretation of the user, with explicit rules for how new evidence reinforces, qualifies, or replaces that interpretation.

DimensionCited user modelsNOBLE
StateLearned user/sequence vectorsHypotheses with conditions, confidence, evidence, and lifecycle
UpdateSequence encoding or interest aggregationEvidence-driven hypothesis revision
AdaptationSequential and temporal interest modelingExplicit refinement and replacement
UseCandidate matching and rankingTask-conditioned hypothesis and evidence retrieval

A temporary restriction and a lasting preference change can both be recent, yet call for REFINE and SUPERSEDE, respectively. NOBLE distinguishes these updates and links them to evidence. We will center the novelty claim on this revision mechanism and its task-conditioned use, with partial observability and non-stationarity as motivation.

W2

Our experiments evaluate whether the maintained beliefs help an agent perform personalized tasks. The belief-update ablation further isolates the contribution of continual revision while retaining hypothesis construction and the downstream reading components. These results support the practical value of updating user interpretations as evidence accumulates.

Direct evaluation of revision correctness would provide complementary evidence about the intermediate representation. Likewise, feeding decomposed evidence into a sequential recommender is a useful way to examine whether the benefits extend to learned user representations. Our current experiments do not establish gains in next-item prediction or personalized content generation; we will make the scope of the empirical claims explicit.

Question 1

NOBLE can be adapted to these tasks by constructing memory from the available user history and using the retrieved hypotheses and evidence to condition a ranking or generation model. Recommendation requires a candidate-scoring interface, while personalized generation can use the memory as additional context. Item-ID-based histories also require an appropriate representation or metadata interface for evidence extraction. Our current evaluation focuses on personalized agent task execution. We have not evaluated NOBLE on these datasets and consider them valuable extensions for assessing transfer to recommendation and generation.

Question 2

Hypothesis revision occurs at inference time. A pretrained LLM Judge assesses new evidence against existing hypotheses, a Resolver executes the revision, and confidence is updated through a deterministic rule. We do not train a separate update policy on user sequences; the evolving state is the external memory.

The belief-update ablation shows that enabling revision improves downstream task success compared with retaining hypothesis construction without continual updates. It supports the utility of the revisions, while a direct revision-quality assessment would additionally require judgments of factual support, applicability, and whether each change appropriately refines or replaces the previous hypothesis. We have not conducted that direct assessment.

Question 3

We thank the reviewer for this valuable suggestion. Our current memory evaluation focuses on downstream task success and question-answering accuracy, without a dedicated ranking-based evaluation. VitaBench 2.0 likewise does not provide a standardized ranking protocol with candidate sets and relevance annotations for computing NDCG, MRR, or Hit@K. These metrics would be valuable for personalized agents, complementing end-to-end success by assessing how well relevant memories or preferred items are ranked. Establishing the corresponding annotations and evaluation protocol is a useful direction for extending the benchmark.

Reviewer_u1T8OpenReview ↗

Response: Clarifying Personalization Claims and Evaluation

Thank you for the detailed response.

W1: Problem Framing

I acknowledge the distinction between latent sequential representations and NOBLE's explicit hypotheses with evidence, applicability conditions, confidence, and lifecycle operations. My concern was that updating user states through ht=f(ht−1,xt)h_t=f(h_{t-1},x_t) is already fundamental to sequential personalization. I welcome the authors' commitment to position the contribution specifically around explicit belief revision rather than the broader notion of non-stationary user modeling.

W2: Is Personalization Quality Directly Evaluated?

I recognize that VitaBench evaluates preference-conditioned task success using ground-truth rubrics, not merely successful API execution. The improvements and belief-update ablations are useful evidence of downstream utility. However, they do not directly isolate whether the explicit belief representation produces better personalized selections than existing user-state models.

Consider a hotel-booking example. Historical interactions indicate that a user prefers Marriott hotels, but a recent interaction reveals that proximity to a hospital is currently more important, while budget constraints remain applicable.

Suppose the environment contains:

  • Hotel A: Marriott, within budget, far from hospital.
  • Hotel B: Non-Marriott, near hospital, within budget.
  • Hotel C: Marriott, near hospital, over budget.

Assume the benchmark rubric identifies Hotel B as the valid choice. NOBLE may REFINE the existing preference and help the agent book Hotel B. However, this result does not establish whether a conventional sequential user representation would also select Hotel B.

Suggested Controlled Evaluation

Since VitaBench already provides histories, candidate environments, and preference rubrics, I suggest evaluating candidate-selection correctness independently of agent execution on suitable subtasks.

  1. Fix identical history HtH_t, current request qtq_t, candidate set Ct\mathcal{C}_t, metadata, and ground-truth rubric RtR_t.
  2. Let a sequential personalization model predict a top-1 candidate ID i^t\hat{i}_t using its learned user representation.
  3. Let NOBLE select an ID using its explicit hypotheses and evidence under the same task conditions.
  4. Deterministically map each predicted ID to its metadata and evaluate both against RtR_t.

For example, define:

1,&\text{if candidate }i\text{ satisfies }R_t,\\ 0,&\text{otherwise}. \end{cases}$$ Then compare $y_{t,\hat{i}_t}$ across approaches. This requires neither a trained ranking head nor booking-agent execution for the sequential recommender. If complete rankings and relevance labels are available, Hit@K, MRR, or NDCG@K could additionally be computed. I understand that not every VitaBench task reduces to one candidate ID. A controlled subset with independently defined candidate relevance would still be informative. Such an evaluation tests rubric-based personalized selection, not prediction of an independently observed future user choice. ### Correctness of Belief Revision A second unresolved question concerns intermediate hypothesis quality. In the same example, NOBLE might correctly REFINE the preference to: > Generally prefers Marriott, except when current trip-specific constraints take priority. Alternatively, it might incorrectly SUPERSEDE the belief with: > No longer prefers Marriott hotels. Both interpretations might lead to booking Hotel B successfully, yet they imply different user states for subsequent tasks. Therefore, I would appreciate direct evidence concerning: - Correctness and factual support of hypotheses. - Accuracy of applicability conditions. - REFINE versus SUPERSEDE decisions. - Handling temporary exceptions versus lasting changes. - Reliability/calibration of confidence updates. The belief-update ablation establishes the usefulness of revisions for downstream success, but not necessarily the correctness of individual revisions. The authors acknowledge that such direct evaluation has not been conducted. ### Current Assessment I recognize the explicit decomposition and positive VitaBench results as meaningful strengths. The rebuttal substantially clarifies the intended positioning, but the experimental concerns remain: **the proposed belief representation is not directly evaluated for correctness or compared with learned sequential representations on controlled personalized candidate selection**. A focused candidate-level experiment using existing rubrics, together with direct revision-quality analysis, would provide evidence more closely aligned with the claimed mechanism. Broader recommendation or generation benchmarks would further establish transfer, although I recognize their adaptation may exceed the rebuttal scope. I therefore **retain my Weak Reject** at this stage and remain open to reconsideration if additional evidence addresses these concerns.
Reviewer_6Gf4OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NOBLE, an evidence-driven memory system for long-term personalized agents. Rather than directly storing or retrieving historical records, NOBLE converts interactions into source-attributed evidence and maintains hypotheses about the user's state within dynamic themes. Each hypothesis contains its content, confidence, supporting and conflicting evidence, applicability conditions, and lifecycle status. New evidence updates existing beliefs through operations such as SUPPORT, REFINE, PROPOSE, and SUPERSEDE. A task-conditioned Planner and Presenter then generate compact context for downstream agents.

Critical strengths
  1. The paper focuses on the non-stationarity of user states and explicitly distinguishes between long-term preferences, temporary exceptions, and information that has been superseded. This is a meaningful improvement over treating long-term memory as simple historical text retrieval.

  2. The combination of evidence, hypotheses, supporting/conflicting evidence, applicability conditions, and lifecycle states provides a clear representation that makes it easier to analyze why the system forms a particular belief about the user.

  3. On VitaBench 2.0, NOBLE outperforms the major baselines across different task models. Moreover, removing belief updating leads to a noticeable performance drop, suggesting that continuously revising user state can indeed be valuable.

Critical weaknesses
  1. NOBLE has the LLM directly output ρ∈[−1,1]\rho \in [-1,1] and a \texttt{weight} ∈[0,1]\in [0,1], which are then accumulated into the support score of a hypothesis. However, the paper does not clearly specify the rules or criteria used by the LLM to produce these confidence values, and the numerical semantics of the confidence scores are also insufficiently defined.

  2. NOBLE consists of multiple LLM stages, including evidence extraction, theme assignment, hypothesis revision, conflict review, consolidation, and presentation. Errors in earlier stages may therefore be propagated by subsequent stages and eventually written into long-term memory.The paper does not investigate whether errors in evidence extraction can be corrected by subsequent updates, whether incorrect \texttt{REFINE} or \texttt{SUPERSEDE} operations can permanently contaminate the memory, how consistent the final memory remains across multiple LLM calls, or how much new evidence is required for the system to recover after an erroneous update. As a result, the long-term reliability of NOBLE remains insufficiently demonstrated.

  3. The paper mainly reports the number of read tokens ultimately passed to the downstream agent and the compression achieved by the Presenter. However, it does not report the total number of end-to-end LLM calls, input/output tokens, latency, or monetary cost, nor does it compare the end-to-end write/update/read cost against other methods.Therefore, the current experiments only demonstrate that the retrieved context can be compact; they do not sufficiently support a broader claim of overall efficiency.

  4. \textbf{RGMem} (ICML2026) and \textbf{Nemori}(ACL 2026) both study the construction and continual updating of higher-level user knowledge from historical content, and therefore substantially overlap with NOBLE's central claims. Omitting these methods raises two concerns: first, it is unclear whether NOBLE's evidence-grounded hypothesis representation is actually superior to existing mechanisms for dynamic memory evolution; second, the novelty of NOBLE in terms of both methodology and empirical evaluation is difficult to assess.At minimum, the paper should include direct experimental comparisons with RGMem and Nemori. If reproducing these methods is infeasible, the authors should provide a rigorous comparison at the methodological, cost, and behavioral levels.

Questions for authors
  1. How exactly are ρ\rho and \texttt{weight} generated? Are few-shot examples, an explicit scoring rubric, discrete score mappings, or additional calibration data used? Please provide the complete prompt and concrete numerical examples, and explain how different types of evidence are mapped to specific scores.

  2. If an error occurs during evidence extraction, theme assignment, hypothesis revision, or consolidation, can NOBLE recover from it? Do you have experiments involving error injection, consistency across repeated runs, or recovery from long-term memory contamination?

  3. Please report the complete number of LLM calls, input/output tokens, latency, and monetary cost for memory generation, updating, conflict review, theme summarization, and retrieval. These measurements should be compared against the baselines under the same quality requirements and with the same downstream model.

  4. Why are RGMem and Nemori not included? Please clarify the substantive differences between NOBLE and these methods in terms of belief/state representation, update mechanisms, and retrieval mechanisms, and provide direct experimental comparisons. I also recommend conducting a unified comparison on other relevant benchmarks, such as PersonaMem (COLM 2025).

The research problem addressed by the paper is important, and the proposed method provides a reasonably interpretable framework. The experimental results are also promising. However, the issues above concern the core confidence mechanism, long-term robustness, efficiency claims, and methodological novelty, and therefore currently prevent me from giving the paper a higher score. If the authors can provide clear answers to these questions, I would consider raising my score.

2 comments

We thank the reviewer for recognizing the strengths of our work and for raising valuable questions that have helped us improve the paper.

Question 1

The Judge performs semantic labeling rather than directly generating numerical scores. Given new evidence and a candidate hypothesis, it assigns a relation label and an evidence-type label. These labels are converted into ρ\rho and ww using the fixed mappings below, with evidence explicitness used directly as the weight. No few-shot examples are used.

RelationSTRONG_SUPPORTSUPPORTWEAK_SUPPORTNEUTRALWEAK_CONFLICTCONFLICTSTRONG_CONFLICT
ρ\rho+1.0+0.7+0.350−0.35−0.7−1.0
Evidence typeExplicit preferenceExplicit constraint or recent changeRepeated behaviorSingle behaviorOther factual observation
ww1.01.00.50.300.15

The support-score increment is Δs=ρw\Delta s=\rho w. For example, a SUPPORT judgment for an explicit preference yields 0.7×1.0=0.70.7\times1.0=0.7, while the same judgment for a single behavior yields 0.7×0.30=0.210.7\times0.30=0.21.

Question 2

NOBLE supports evidence-driven recovery by preserving evidence provenance and revisable hypotheses. When new evidence contradicts a hypothesis, conflict review reassesses the proposed revision: REFINE corrects its content or applicability conditions, while SUPERSEDE replaces an incorrect hypothesis. Subsequent consolidation can update the theme summary using the corrected hypotheses.

We injected an incorrect preference that the user accepts the taste of passion fruit, despite the user explicitly stating, “Okay, but please do not recommend passion fruit to me in the future.” We then replayed a subsequent interaction in which the user stated, “I really dislike the taste of passion fruit. Please do not recommend anything related to it.” This explicit rejection provides corrective evidence that can trigger conflict review and SUPERSEDE the erroneous preference, followed by a theme summary update during consolidation.

Recovery requires corrective evidence to reach the affected hypothesis. Extraction omissions or theme-assignment errors may require reprocessing the relevant history. Beyond individual error-injection case, we have not yet conducted a systematic evaluation of error recovery or long-term memory contamination.

Question 3

We report write costs averaged per interaction and read costs averaged per query.

MethodWrite callsWrite inputWrite output (k tokens)Write time (s)Read callsRead inputRead output (k tokens)Read time (s)
SimpleMem0.81.1k0.65536.24.33.7k0.44644.0
A-MEM7.66.7k0.787.00001.5
PlugMem2.73.1k0.6142.823.3k0.9431.6
NOBLE3.23.8k0.5450.223.5k0.9334.8

NOBLE's stage-level token consumption is shown below.

TokensEvidence extractionBelief updatingConflict reviewStable consolidationPlannerPresenter
Input1.4k1.0k0.8k0.56k1.0k2.5k
Output1801609490120810

NOBLE's write overhead arises from maintaining explicit, revisable user hypotheses. Beyond evidence extraction, belief updating evaluates new evidence against existing hypotheses, conflict review checks proposed revisions, and consolidation refreshes stable theme summaries. These operations require additional LLM calls and repeated processing of evidence and hypothesis context. Memory writing can also run asynchronously or offline, separately from task-time retrieval.

Question 4

Thank you for highlighting these works. Their omission was an oversight. RGMem evolves hierarchical user profiles through threshold-triggered aggregation and retrieves information across episodic and graph levels. Nemori maintains episodic and semantic memories, distills knowledge through predict–calibrate learning, and retrieves memories by vector similarity.

NOBLE explicitly represents individual hypotheses with applicability conditions, confidence, supporting/conflicting evidence, and lifecycle status. Its update actions distinguish contextual refinement (REFINE) from replacement (SUPERSEDE). Retrieval uses theme-level planning to select hypotheses and evidence, including evolving hypotheses, and preserves their applicability and uncertainty in task-specific context.

We have not completed direct comparisons with these methods or additional PersonaMem experiments within the rebuttal period. We plan to add these evaluations under a unified protocol in subsequent work.

Thank you for the response and additional analyses. The clarification of the confidence-scoring procedure and the detailed cost breakdown address part of my concerns and improve the transparency of the method.

However, my main concerns about long-term reliability and methodological novelty remain. The recovery example illustrates that an incorrect hypothesis can be corrected when explicit contradictory evidence arrives, but it provides limited evidence about robustness to error propagation during repeated memory updates. The authors also acknowledge that systematic recovery evaluation has not been conducted.

The discussion of RGMem and Nemori helps clarify the intended differences. However, without direct comparisons or targeted evidence isolating the benefits of NOBLE’s distinctive design choices, its added value over closely related approaches remains insufficiently established.

I continue to appreciate the interpretable framework and promising results on VitaBench 2.0. Nevertheless, the rebuttal does not sufficiently resolve the central concerns underlying my original assessment. I therefore maintain my rating.

Reviewer_bpzdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NOBLE, a memory system for personalized LLM agents under partial observability and non-stationary user states. The core idea is to move away from static record retrieval by maintaining dynamic hypotheses with confidence scores that evolve through atomic evidence. Interactions are parsed into evidence items within dynamic themes, updated using five explicit lifecycle actions, and consolidated into theme cards. At task time, a Planner and a Presenter build a compact context for the downstream agent. The authors evaluate NOBLE on VitaBench 2.0 using three backbone models and on LongMemEval, demonstrating solid gains in task success rates and user preference updates.

Critical strengths
  1. In my opinion, treating personalized memory as an evolving belief state rather than static text records is an effective and well-motivated approach for non-stationary environments. In particular, separating contextual exceptions in REFINE from permanent replacements in SUPERSEDE provides a clean solution to recency and frequency issues.

  2. Empirical results on VitaBench 2.0 show consistent gains over competitive baselines across multiple backbone models. The improvement on the Update task is substantial, and the ablation study confirms that active belief updates drive these performance gains.

  3. This paper is well-written and easy to follow.

Critical weaknesses
  1. This paper only reports read token counts in Figure 4 and omits the computational cost and wall-clock latency of the write pathway. Maintaining memory requires multiple sequential LLM calls across the Extractor, Judge, Conflict Review, and Consolidation steps, which could make online operation expensive and slow compared to simpler baselines.

  2. The evaluation lacks statistical variance reporting. Table 1 and Table 2 present mean success rates over four runs without standard deviations, which makes it difficult to assess the statistical significance of the reported improvements.

  3. The authors do not provide a sensitivity analysis for key operational thresholds. Important parameters such as the confidence threshold τ=0.65\tau=0.65, the support evidence count m=3m=3, and the consolidation interval F=3F=3 appear to be hand-tuned without ablation.

  4. NOBLE shows a noticeable performance drop on the Single-Session Preference task in LongMemEval compared to SimpleMem. This suggests that the hypothesis lifecycle may create unwanted inertia when an agent needs to adopt immediate, one-off user preferences.

Questions for authors
  1. What is the total token consumption and runtime latency of the write pipeline per interaction turn, and how does this overhead compare directly with baselines like A-MEM, SimpleMem and other baselines?

  2. Could you report the standard deviations across the four runs in Table 1 and Table 2 to substantiate the statistical significance of the gains? I believe the authors should report the standard deviations for at least three seeds (ideally, five seeds)

  3. How sensitive is system performance to the stability threshold τ\tau, the minimum evidence threshold mm, and the consolidation interval FF?

  4. What causes the accuracy drop on the SS-Preference split in LongMemEval, and how can the system quickly adopt temporary preferences before they satisfy the stability gate?

2 comments

We thank the reviewer for the constructive suggestions, which have helped us clarify the method and strengthen the empirical analysis.

Question 1

NOBLE consumes 3.8k input tokens and 540 output tokens per interaction, with an average write time of 50.2s. This includes extraction, theme assignment, hypothesis revision, conflict review, and consolidation. The following comparison reports details.

Methodwrite inputwritw outputwrite timeread inputread tokenread time
SimpleMem1.1k65536.2s3.7k44644.0s
Amem6.7k71387s02.8k1.5s
plugmem3.1k61042.8s3.3k94031.6s
ours3.8k54050.2s3.5k84634.8s

NOBLE involves multiple LLM calls during memory writing, which incur token and latency costs. Nevertheless, its overall write cost and latency remain comparable to those of the evaluated baselines. All methods use the same API-based inference setup. Serving Qwen3-32B locally with an optimized inference configuration could further reduce latency. The write pipeline can also run asynchronously in the background, allowing memory construction to be scheduled separately from task-time retrieval.

Question 2

Below, we report mean ± standard deviation across four evaluation runs for the main results and ablations in Tables 1 and 2. NOBLE achieves the highest mean overall success rates, with gains exceeding the observed run-to-run variation. These results support the robustness of NOBLE’s performance improvements.

ModelMethodDeliveryInstoreOTAAvg@4
gpt-4.1Full Context31.9 ± 2.029.1 ± 1.212.5 ± 2.727.5 ± 1.1
MemAgent39.1 ± 1.534.5 ± 2.311.8 ± 1.931.8 ± 1.6
RAG30.2 ± 1.732.7 ± 1.313.2 ± 2.428.6 ± 1.2
SimpleMem33.7 ± 2.233.8 ± 1.923.5 ± 2.831.6 ± 2.1
A-MEM37.11 ± 1.435.4 ± 2.021.0 ± 2.333.7 ± 1.6
PlugMem34.2 ± 2.334.1 ± 1.123.0 ± 3.432.1 ± 1.3
NOBLE48.8 ± 1.737.0 ± 1.228.5 ± 2.641.0 ± 0.9
MethodDeliveryInstoreOTAAvg
ours48.8 ± 1.737.0 ± 1.228.5 ± 2.641.0 ± 0.9
w/o presenter44.8 ± 2.538.6 ± 1.824.3 ± 2.739.2 ± 1.5

| w/o plannner | 40.6 ± 1.2 | 32.4 ± 1.3 | 20.4 ± 1.9 | 34.7 ± 1.1 | | w/o belief | 35.9 ± 1.9 | 32.1 ± 1.5 | 21.0 ± 2.4 | 32.8 ± 1.6 |

Question 3

We assess sensitivity by varying each parameter while holding the other two fixed, following the evaluation protocol in the manuscript. The default values were selected through a grid search on a small subset of the data and kept fixed across all experiments. These defaults do not always yield the highest performance in this sensitivity analysis. For example, setting m=2m=2 slightly improves the average success rate, indicating that the stability criterion affects how hypotheses are incorporated into stable belief summaries. Overall, performance varies modestly across the tested settings, suggesting that NOBLE is not highly sensitive to these parameters within the evaluated ranges.

τ\tauDeliveryInstoreOTAAvg
0.649.337.726.441.6
0.6548.837.028.541.0
0.747.838.326.741.3
mDeliveryInstoreOTAAvg
249.536.130.642.2
348.837.028.541.0
446.334.620.838.3
FDeliveryInstoreOTAAvg
246.838.622.239.8
348.837.028.541.0
447.039.029.841.5

Question 4

NOBLE achieves 60.0% on SS-Preference versus SimpleMem’s 80.0%. We noted that this split contains 30 questions, so the 20% gap only corresponds to 6 answers.

The stability gate controls inclusion in consolidated summaries and affects presentation tiers. Hypotheses below the gate and their supporting evidence remain retrievable, allowing preferences to inform an answer before becoming stable user knowledge. The gap may arise when specific preference cues receive insufficient emphasis during retrieval or context compression.

For example, one SS-Preference question asks for video-editing learning resources. The reference answer expects resources tailored to Adobe Premiere Pro and its advanced settings, based on the user’s expressed interest in a historical session. This explicit cue can directly guide the recommendation without requiring repeated confirmation. NOBLE’s direct-evidence retrieval provides this path; preserving such specificity through retrieval and presentation is essential for successful preference use.

Thank you for the author's response. I carefully checked the responses and they resolved all the concerns that I had. so I increased my score to Weak Accept. I would suggest adding these results to the camera ready version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.