acceptodds
Under review as a conference paper at ICLR 2027

How Can Recommendation Feedback Evolve Agent Memory?

Abstract

Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution. However, these signals are delayed and noisy, confounded by audience composition, placement, and recommendation policies, and may result from the combined influence of multiple memories, making accurate attribution difficult. Existing methods rely primarily on immediate feedback or semantic retrieval and therefore struggle to reliably translate recommendation outcomes into memory fitness. To address this challenge, we propose TIDE (Trajectory-Informed Directed Memory Evolution), an external memory evolution framework driven by delayed recommendation feedback. We further introduce Memory Evolution Gain (MEG), which measures the utility improvement of evolved memory over an initially frozen memory baseline on future tasks. TIDE treats memory as a capacity-constrained population of experiences: temporal and semantic credit assignment estimates contextual fitness, while responsibility credit distributes outcome signals according to the memories referenced during generation. These signals are then used to reinforce, crossover, mutate, or evict memories. On an e-commerce membership marketing content-generation agent, TIDE achieves a +7.75-percentage-point MEG in offline temporal replay and significantly improves both unique click-through rate (UCTR) and activation rate in an online A/B test. On a delayed-label benchmark , TIDE achieves the lowest mean absolute error (MAE) and root mean squared error (RMSE) and the highest MEG among the compared methods, demonstrating its effectiveness.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KMMROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LLM agents are used to generate advertising content, which are then shown in production and accumulate various kinds of feedback that might be used as a signal to improve the system. Although a data-rich environment, this is a difficult problem as it requires credit assignment over time and materials. To address this challenge, the authors of this paper propose TIDE. TIDE is a training-free method for managing an external memory store that will be given in-context to the generator. TIDE relies on an evolutionary prompting pipeline to manage a capacity-constrained population of experiences. In real-world deployment A/B tests, TIDE improves over baselines in click-through and activation rate.

Critical strengths

I think that this problem setting is very interesting, and understudied relative to the more static and structured environments of many continual learning and memory papers. I really like the aspect of the formulation where “the value of a memory update is determined by subsequent tasks rather than by the example that triggered the update”; this represents an extremely important and tough to crack class of problems.

Critical weaknesses

TIDE is heavily engineering-based and heuristic. The methods section mixes business rules and machine learning methods, and it is unclear what general principles can be extracted and applied to other tasks of interest to the community. No motivation is given for the particular solution; for example, why the method is training-free when there is plentiful historical data, or why an evolutionary approach might be appropriate. This issue extends to the MEG metric. The experiments on MemoryCD seem like they could help demonstrate the generality of the system, but that benchmark is never described or illustrated so that doesn’t happen.

I also wonder whether the results in Table 1 are significant and/or reliable. They seem to rely on an unvalidated LLM judge.

Questions for authors
  • How does this solution map onto problems outside of this domain?
  • Why is training-free best, when it seems like you could collect data here and fine-tune some of these parts?
  • Why is genetic approach the right/principled approach to take here?
7 comments
AuthorsOpenReview ↗

Generalizability concerns

We sincerely thank the reviewers for recognizing that our research is innovative, interesting, and addresses a current research gap. To the best of our knowledge, TIDE is among the earliest to systematically study the self-evolution of content-generation agents driven by delayed and context-dependent recommendation feedback.


We note the reviewer’s focus on generalizability. Like the reviewer, we are also committed to providing new insights into agent self-evolution through our research. Below are our responses to the issues regarding generalizability:


W1 / Q1: Generalizability: How can our approach be utilized by others?

1. From the Perspective of Application Scenarios

Our framework can be directly applied to recommendation and similar scenarios. For other tasks, the framework is structured into two layers:

  • Layer 1: Domain-Specific Feedback Construction. Different domains require specific definitions for feedback maturation conditions, context variables, and task metrics. For instance, coding scenarios need to consider test coverage, dependency environments, and code reviews, while personal assistants must consider user confirmation and actual task completion. However, the mechanisms for these signals can consistently be constructed from temporal, spatial, and logical dimensions.
  • Layer 2: Domain-Independent Memory Evolution Logic. This includes the pipeline of "Feedback Maturation — Context Decoupling — Credit Assignment — Memory Pool Evolution — Future Task Validation." This entire process is reusable. MEG handles the final step: determining whether a memory update truly improves performance on strictly future tasks. A personal assistant could use task completion and user correction rates, while a coding agent could use task resolution and regression rates.

In summary, as long as a task's feedback cannot be instantly attributed and summarized into memory like a simple math problem, our logic is applicable. As shown in Figure 1, the CREATE, SERVE, and LEARN components form a closed loop, corresponding to LLM output, feedback acquisition, and memory updates. To put it simply: as long as an LLM can generate output, receive feedback, and continue to optimize based on it, our framework can be utilized.

2. From the Perspective of Application Methods

As explained in W2, the memories evolved via TIDE can be used not only directly during the inference stage but also to help construct training data. Experimental results show that incorporating these memories during the training phase improves the efficiency and diversity of the generated content. We believe this is because evolved memories enrich prompt templates and experience expressions in the training data, reducing the model's reliance on fixed sample patterns and thereby enhancing training diversity and robustness.


W4: Clarification on Generalization Experiments

We have actually introduced the MemoryCD benchmark in the main text (as shown below), and we detailed the specific modifications in the "Public delayed-label benchmark" section of the Appendix.

Evo-Memory, EvoMemBench, and MemoryCD evaluate such mechanisms under continual tasks, evolving interaction histories, and long-term user behavior (Wei et al., 2026; Wang et al., 2026; Zhang et al., 2026).

We appreciate your reminder that the citations in the experimental section were not prominent enough. We will add the corresponding references accordingly 🔧🔨.

AuthorsOpenReview ↗

W2 / Q2: Why is it training-free?

W2 / Q2: Why is it training-free?

Thank you for your interest in the efficiency of our method. We believe this choice depends on the LLM "brain" behind the agent: since our service utilizes LLMs larger than 27B, a training-free approach is a highly efficient choice. We have also discussed and explored in the main text whether training-free methods perform worse than training-based ones:

First, theoretically, our investigation in the Introduction, Appendix, and Related Work sections shows that training-free methods possess immense potential for agent evolution.

Second, in empirical comparisons, Section 4.2 contrasts TIDE with sample-score-based fine-tuning methods such as BERT and Distilled Recommenders. Although BERT scores higher on certain pre-release prediction metrics, its advantages fail to generalize to future tasks, showing a tendency to overfit historical score targets. In contrast, TIDE achieves the highest Future MEG (+0.180) and win rate (60.7%), along with the lowest negative transfer rate (25.0%). This demonstrates its robustness in converting feedback into memory updates that are effective for future generation.

Third, TIDE can also be combined with training. Table 1 shows that combining TIDE with parameter backpropagation yields better results than backpropagation alone. We further compared the effects of introducing TIDE memories during the training phase versus the inference phase: inference-stage injection performs better on fundamental metrics like compliance and formatting, likely because raw sample signals have a stronger influence on these metrics than memory. Training-stage injection, however, excels in content efficiency and diversity. We believe TIDE memories indirectly enrich the prompt information in training samples, making the model's output more aligned with actual production needs rather than simply memorizing training samples.

MethodCompliance ↑Format ↑Efficiency ↑Diversity ↑
Training55.067.540.065.0
Inference70.075.037.552.5

Table: TIDE combined with backpropagation; experimental settings follow Table 1 in the main text.

In summary, for models with large enough parameter scales to understand memory, a training-free approach is the most cost-effective choice. For smaller models, we suggest synergistically integrating TIDE memories into the training and inference pipeline.

AuthorsOpenReview ↗

W3 / Q3: Why Choose an Evolutionary Design?

W3 / Q3: Why Choose an Evolutionary Design?

We appreciate the reviewer raising this pivotal question. Our choice of an evolutionary design is primarily dictated by the structural characteristics of the problem itself.

First, we optimize a set of discrete, interpretable natural language memories rather than continuous, differentiable model parameters. Feedback in recommendation tasks is often delayed, context-dependent, multi-objective, and non-differentiable; consequently, there is no "target memory" that can be derived directly via gradients. More importantly, as you noted, the value of a memory update can only be validated by its performance on subsequent tasks, rather than the current sample that triggered the update. Therefore, directly overwriting memory based on a single feedback instance risks erroneous updates, while simple appending leads to uncontrolled memory bloat. Maintaining multiple candidates supported by different evidence and selecting them based on future feedback within a fixed capacity is a more robust approach.

Second, preliminary evidence is provided in Appendix Table 4. After ten rounds of feedback, TIDE achieved the highest Mean Utility (0.809) and Joint Pass (0.617) while consuming only 2,121 memory tokens—fewer than Full History, Append-only, and Random Replacement. Notably, 9 out of 10 "EVICT" operations in TIDE correctly targeted injected noisy memories. This demonstrates that TIDE’s performance gains stem from selective evolution and denoising rather than the mere accumulation of historical information.

Third, inspired by your question, we further investigated other memory evolution methods: ReasoningBank [1] (extracting reusable strategies from successes and failures), KARMA [2] (generating local atomic modifications and using a rollback mechanism), and EvolveR [3] (distilling interactions into general principles). We have added a comparative experiment against these methods, which we will report in the Appendix.

MethodMean Utility ↑MEG ↑Joint Pass ↑Negative Transfer ↓
Frozen Memory0.83530.000013.3%0.0%
ReasoningBank-style [1]0.8522+0.016913.3%36.7%
KARMA-style [2]0.8572+0.02193.3%33.3%
EvolveR-Memory-style [3]0.8222-0.013116.7%50.0%
TIDE0.8600+0.024716.7%33.3%

Table: Comparison of memory evolution methods. We conducted a targeted experiment across 30 tasks with three random seeds (13, 42, 97) over 50 evolution rounds. While each method managed its own memory context, all shared the same generator, task inputs, and output requirements. We evaluated results using a frozen DeepSeek evaluator. Mean Utility averages Compliance, Format, and Efficiency, while Joint Pass also incorporates Diversity. TIDE achieved the highest Mean Utility (0.8600) and MEG (+0.0247), while matching the best results in Joint Pass and Negative Transfer, further justifying our evolutionary design.

References

  1. Ouyang et al. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. ICLR 2026.

  2. Miao et al. KARMA: Training-Free Knowledge-Atomic Harness Evolution for Industrial Advertising Agents. 2026.

  3. Wu et al. EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle. ICML 2026.

Reviewer_2rvaOpenReview ↗
Rating
2
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces TIDE, a training-free framework that updates an agent’s external memory using delayed recommendation feedback while keeping the generator fixed. It combines temporal credit to assess feedback reliability, semantic credit to interpret outcomes in context, and responsibility credit to identify which memories influenced generation. These signals guide memory reinforcement, revision, merging, and eviction under a capacity limit. The paper evaluates TIDE through offline replay, a one-week randomized online A/B test, and a delayed-label adaptation of MemoryCD. TIDE achieves an offline Memory Evolution Gain of +7.75 points relative to no memory, reports relative online improvements of 5.10% in UCTR and 4.79% in activation rate, and obtains the best prediction errors among the compared MemoryCD methods.

Critical strengths
  1. TIDE addresses a practical problem: turning delayed and context-dependent recommendation outcomes into useful memory updates without retraining the generator. Separating feedback reliability, contextual interpretation, and memory responsibility provides a clear way to organize this process.
  2. The evaluation combines several complementary forms of evidence. Offline replay tests performance on future requests, controlled experiments examine responsibility attribution, and the randomized online experiment measures actual engagement and activation outcomes.
Critical weaknesses
  1. The evidence for the full three-stage design is weaker than the evidence for responsibility credit alone. Removing temporal credit, semantic credit, or individual editing operators lowers mean utility, but all corresponding confidence intervals include zero. These results do not show that the components are ineffective, but they leave their independent benefits uncertain.

  2. The key offline conclusions rely on relatively small evaluations and a single LLM judge. The main experiment uses 50 future requests, the responsibility study uses 20 controlled requests, and several additional studies use 12 tasks with one seed. Using the same judge to guide edits and assess future utility also leaves open whether the gains transfer to other evaluators.

Questions for authors

1.Could you specify how matched controls are selected, how contextual fitness is computed, and how candidate memories are ranked, promoted, or rejected? Matching-quality checks or sensitivity analyses would also help clarify how much residual confounding may remain in the semantic credit signal.

2.Could you test temporal credit, semantic credit, and the editing operators on more tasks or repeated runs, particularly under conditions where each component is expected to help?

3.Could you report confidence intervals for the main paired comparisons, repeat selected experiments across independent runs, and validate a subset of outputs with human reviewers or a different LLM judge?

5 comments
AuthorsOpenReview ↗

W1/Q3: Confidence of Appendix Table 5

We sincerely thank the reviewer for recognizing that "our problem definition has practical value; the method is clearly organized; and the evaluation evidence is rich." Thank you for your careful and thorough review. We also noted the reviewer's concern about our experiment scale and the seeds in the Appendix experiments. Our response is below.


W1/Q3: Confidence of Appendix Table 5

As Reviewer 4rPp also found, the results in Table 5 are indeed not significant enough, with only ResponsibilityCredit showing a significant advantage. We believe this is because the evaluation scale was not large enough, the number of seeds and rounds was insufficient, leading to high variance in the results, and the initial setup already contained a large batch of memories produced by TIDE, which made the ablation of memory-related operators appear to have little effect. We therefore added experiments with a larger scale and more seeds, plus an initial setup with a small amount of random memory, to replace Table 5.

VariantRound-20 Utility ↑Δ vs. Full ↓95% CI of ΔUtility AUC ↑
Full TIDE0.6930.000[0.000, 0.000]0.629
w/o Temporal Credit0.650-0.043[-0.083, -0.022]0.586
w/o Semantic Credit0.600-0.093[-0.167, -0.050]0.572
w/o Responsibility Credit0.639-0.054[-0.092, -0.013]0.575
w/o REVISE0.654-0.039[-0.109, -0.004]0.603
w/o MERGE0.458-0.235[-0.272, -0.225]0.528
w/o EVICT0.496-0.197[-0.258, -0.163]0.537

Table 5: Credit and memory-operator ablations after 20 feedback rounds. Mean Utility is the average of normalized Compliance, Format, and Efficiency scores. All conditions use the same initial memory, feedback sequence, memory capacity, evaluation tasks, and 5 random seeds. Differences and 95% confidence intervals are computed via paired task-level bootstrap relative to Full TIDE. Utility AUC is computed over rounds 1–20. We can see that each ablation demonstrates the effectiveness of our components. After modifying the initial memory setup, the utility of the memory-related operators becomes clearly evident.

AuthorsOpenReview ↗

W2 / Q2: Evaluation Scale and Setup

W2 / Q2: Evaluation Scale and Setup

Regarding evaluation scale: the main experiments use 50 future requests, which is quite substantial compared to real-world weekly update iterations. Some appendix experiments use 12 tasks with only a single random seed; we acknowledge this limitation, which is also why the confidence levels in Appendix Table 5 are not significant. We have therefore added larger-scale, multi-seed experiments to Table 5 and reported confidence intervals, as shown in W1.

Regarding the choice of LLM judge, which Reviewer KMMR also raised: first, the evaluator used in our main experiments is unified, ensuring fair evaluation. Second, we supplemented evaluations using different LLMs. As the per-dimension results below show, some evaluators exhibit noticeable systematic bias. For example, Gemini 3.1 Flash-Lite and Claude Opus 4.7 rate Diversity markedly more optimistically than human reviewers, while Claude Opus 4.7 and GPT-5.6 Sol also score Efficiency noticeably higher. In contrast, DeepSeek-V4-Pro differs from human review by only 1.9, 1.0, and 3.2 points on Compliance, Format, and Diversity, respectively—closer to human judgment overall, without pronounced optimism bias on any dimension. We therefore consider DeepSeek-V4-Pro the relatively better-aligned choice among the compared evaluators. In the revision, we will add this blind evaluation setup and cross-model comparison to the appendix to further justify our evaluator choice.

Judge ModelComplianceFormatEfficiencyDiversity4-Dim. Avg.
GPT-5.6 Sol86.875.685.855.775.9
GPT-6 Astra91.776.384.355.376.9
Gemini 3.1 Flash-Lite95.775.791.475.784.6
Claude Opus 4.782.682.880.468.078.4
DeepSeek-V4-Pro86.783.378.351.775.0
Human Review84.884.369.748.571.8

Table: We sampled 50 items from each of 3 channels (150 total) for blind evaluation, with four independent LLMs scoring Compliance, Format, Efficiency, and within-channel Diversity on a unified 100-point scale; average scores per dimension are reported.

AuthorsOpenReview ↗

Q1: Memory Selection, Ordering, and Rejection Mechanism

Q1: Memory Selection, Ordering, and Rejection Mechanism

We thank the reviewer for their interest in our memory mechanism. Sections 3.3–3.4 and Algorithm 1 of the original submission already specify matched controls, Semantic Credit, Responsibility Credit, and the overall GovernAndSelect pipeline. Specifically:

Selection: For each target material that has passed the Temporal Credit check, we select matched controls from historical or contemporaneous traffic. Control materials must match the target on key delivery conditions, including product, audience, campaign stage, placement, and recommender version. Semantic Credit takes effect only when the number of controls and their consistency are sufficient to support comparison; otherwise, the record remains invalid and does not participate in feedback-driven memory updates. This gating mechanism avoids credit assignment under clearly incomparable traffic conditions.

Fitness: Contextual Fitness measures the target material's multi-objective performance relative to matched controls, rather than its absolute business metrics. For example, if both the target and control materials show a CTR increase during a shopping-festival warm-up period, this common shift is treated as environment-related; only the target's relative advantage under identical conditions—and only when this advantage can be tied to specific content decisions—constitutes positive evidence. Metrics such as CTR and CVR are retained as a multi-objective vector; when relative performance is unstable, objective conflicts cannot be explained, or counterexamples remain unresolved, Semantic Credit refuses to produce fitness usable for updates.

Rejection: Finally, candidate memories undergo a governance process of "evidence gating—responsibility attribution—capacity selection." Only memories that actually participated in generation, are semantically consistent with the target content decision, and would change the corresponding decision under counterfactual removal receive Contextual Fitness. Revised, merged, or newly generated memories first enter as provisional candidates carrying full provenance, applicable context, and version information. The system then performs selection through validity checks, deduplication, conflict handling, expiration checks, and capacity constraints. Candidates that are consistently supported by evidence and have unique applicable value are promoted; candidates lacking responsibility evidence, duplicating existing memories, contextually incompatible, persistently low-fitness, or contradicted by subsequent feedback are rejected, further narrowed, or eliminated. Thus, the ordering here is not a single similarity ranking but a hierarchical decision: first, whether the feedback and matching evidence are valid; second, whether the memory is genuinely responsible for the output; and only then, among candidates passing the first two layers, a capacity-constrained selection based on cumulative fitness, evidence reliability, redundancy, and scenario coverage value.

Regarding "matching-quality checks or sensitivity analyses would also help clarify how much confounding may remain in the semantic credit signal": we suspect you may be concerned that our credit-filtering mechanism contains noise—precisely an issue we focused on and valued in our study. However, since the future has not yet arrived, there is no ground-truth reference. We therefore chose to address this question in Section 4.2, RQ2. In brief: we compared memories produced by our signal pipeline against directly training BERT on high/low-score samples and against directly distilling and using downstream recommendation models, and found that our method performs best (with the least noise), because our method's subsequent replacement of responsibility memories handles this noise well.


We sincerely thank the reviewer for their insightful and professional feedback. By refining the confidence values in Appendix Table 5, clarifying the evaluator design, and detailing the memory settings, we have addressed all concerns regarding rigor and reproducibility. (Note: As you suggested, our planned PDF revisions are as follows and will be uploaded before the rebuttal deadline: 1. updates to Appendix Table 5; 2. the evaluator experiments added to the appendix.) We believe these revisions strengthen our work and look forward to your positive feedback. We welcome further discussion at any time.

Reviewer_4rPpOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper focuses on memory evolution for content-generation agents in recommendation scenarios. Within a continuous feedback loop involving content generation, recommendation and delivery, and metric feedback, the authors propose TIDE (Trajectory-Informed Directed Memory Evolution), a training-free framework that transforms recommendation outcomes into memory-level fitness through three core credit-assignment: (1) temporal credit, which evaluates the reliability of delayed feedback; (2) semantic credit, which distinguishes content effectiveness from environmental influences; and (3) responsibility credit, which identifies influential memories using usage traces and counterfactual replay. Based on these signals, TIDE evolves a capacity-constrained memory population through four operations: reinforcement, crossover, mutation, and eviction. The authors validate TIDE through offline experiments, online A/B testing, and a public delayed-label benchmark, demonstrating consistent improvements in memory evolution.

Critical strengths
  1. Practical motivation and good solution for feedback attribution. The problem addressed in this paper is highly relevant to real-world industrial applications. To leverage noisy recommendation feedback to guide agent memory updates, the authors propose three credit assignment mechanisms that form a comprehensive and well-reasoned framework, making the proposed approach convincing.
  2. Comprehensive experiments and Detailed analysis of memory evolution under capacity constraints. The authors evaluate TIDE through offline temporal replay, online A/B testing, controlled attribution experiments, and a public delayed-label benchmark. The inclusion of a real-world online A/B test strengthens the practical relevance of the proposed method. Beyond various experiments, the paper investigates memory-operator ablations, memory diversity, etc. These analyses provide useful insights into how TIDE maintains a bounded memory population while improving future generation quality.
Critical weaknesses
  1. The contributions of several proposed components are not statistically convincing. According to Table 5 in Appendix C, removing Temporal Credit, Semantic Credit, REVISE, MERGE, or EVICT leads to relatively small performance degradations, with confidence intervals that include zero. Only the removal of Responsibility Credit produces a statistically detectable decrease. This raises questions about whether all proposed components are essential to TIDE's effectiveness.
Questions for authors
  1. How essential are Temporal Credit and Semantic Credit to the overall framework? As mentioned in the Critical Weaknesses, Could the authors provide additional experiments or explanations to demonstrate their necessity?
  2. Could the validity filtering in Temporal Credit introduce selection bias by favoring materials with mature and stable feedback trajectories? In other words, are materials with low exposure, long conversion cycles less likely to contribute to memory evolution?
  3. How does TIDE handle conflicting objectives during memory selection? For example, if a memory improves CTR but reduces CVR, how is its overall fitness determined?
  4. Does the memory capacity constraint itself affect TIDE's performance?
3 comments

First, we sincerely thank the reviewer for the recognition that "our problem definition has practical value; the method is clearly organized; and the evaluation evidence is rich." We also thank the reviewer for their careful and thorough review. We have noted the reviewer's concern regarding the scale of our experiments and the experimental seeds in the appendix, and our response is provided below.


W1/Q3: Confidence of Table 5 in the Appendix

As Reviewer 4rPp also observed, the experimental results in Table 5 are indeed not sufficiently significant, with only ResponsibilityCredit showing a significant advantage. We believe this is because the evaluation scale was not large enough and the number of seeds and rounds was insufficient, leading to greater variability in the experimental results. Moreover, the initial setup already contained a large batch of tide-produced memories, which made the ablations of memory-related operators appear to have little effect. We therefore added experiments with a larger scale and more seeds, along with an initial setup with a small amount of random memory, to replace Table 5.

VariantRound-20 Utility ↑Δ vs. Full ↓95% CI of ΔUtility AUC ↑
Full TIDE0.6930.000[0.000, 0.000]0.629
w/o Temporal Credit0.650-0.043[-0.083, -0.022]0.586
w/o Semantic Credit0.600-0.093[-0.167, -0.050]0.572
w/o Responsibility Credit0.639-0.054[-0.092, -0.013]0.575
w/o REVISE0.654-0.039[-0.109, -0.004]0.603
w/o MERGE0.458-0.235[-0.272, -0.225]0.528
w/o EVICT0.496-0.197[-0.258, -0.163]0.537

Table 5: Credit and memory-operator ablations after 20 feedback rounds. Mean Utility is the average of the standardized Compliance, Format, and Efficiency scores. All conditions use the same initial memory, feedback sequence, memory capacity, evaluation tasks, and 5 random seeds. Differences and 95% confidence intervals are computed via a paired task-level bootstrap method relative to Full TIDE. Utility AUC is computed over rounds 1–20. We can observe that: each ablation demonstrates the effectiveness of our respective components. After modifying the initial memory setup, the utility of the memory-related operators becomes clearly evident.

AuthorsOpenReview ↗

Details of the Memory Mechanism

We thank the reviewer for their interest in and curiosity about our memory mechanism. Below are our responses regarding the memory mechanism.

Q2 Selection Bias

We thank the reviewer for their attention to our temporal credit. It sounds as though Temporal Credit's validity filtering would make materials whose feedback has already matured more likely to participate in the current round of memory evolution. Correspondingly, materials with low exposure or long conversion cycles may still be in a pending state, and thus contribute relatively little at the current update point.

In fact, however, in Section 3.3 of the original paper, we perform relative comparisons using contemporaneous or historically matched traffic, controlling for item, audience, campaign stage, placement, and recommender version. As a result, low-exposure, long-conversion-cycle materials are compared only with contemporaneous, same-channel materials with similar feedback cycles, and do not incur systematic bias from direct comparison with high-exposure, short-cycle materials.

Turning to your question: are low-exposure, long-conversion-cycle materials less likely to contribute to memory evolution? Our answer is that [low-exposure materials do not fail to participate in memory evolution; they participate later.] At the same time, as Reviewer KMMR agreed, "the value of memory updates is determined by subsequent tasks, not by the samples that trigger the update." If one looks only at snapshot metrics of samples rather than at Temporal Credit, it is precisely low-exposure materials that would be more likely to be buried.

Q3 Handling Conflicting Objectives

We thank the reviewer for their interest in our memory conflict handling. TIDE does not immediately weight CTR and CVR into a single "overall fitness score." As described in Section 3.2, Temporal Credit retains the results of each objective and their uncertainties, forming a multi-objective fitness vector, and decides how to update memory based on whether the conflict can be explained by context.

Take the example of a memory that increases CTR but decreases CVR. TIDE first uses Semantic Credit to compare the two metrics separately against contemporaneous matched controls, and checks whether the difference can be explained by audience, campaign stage, placement, or the recommendation environment. If the conflict can be explained by context—for example, a certain phrasing increases clicks in acquisition scenarios but decreases CVR in conversion-oriented scenarios—TIDE does not simply judge the memory as "good" or "bad," but instead narrows its scope of applicability through REVISE or SPLIT, forming a memory with explicit scenario conditions: the experience is retained in scenarios that support CTR, while recording that it does not apply to conversion-oriented scenarios. If the conflict between CTR and CVR cannot be explained by the existing context and evidence, Semantic Credit in Section 3.3 marks that evidence as invalid, i.e., sets zisem=0z_i^{sem}=0. This feedback does not drive reinforcement, modification, or eviction of the memory, but instead waits for subsequent evidence to further resolve the conflict.

Q4: Memory Capacity

We thank the reviewer for their attention to memory capacity. In fact, we have already specifically studied memory evolution under capacity constraints in Appendix B / Figure 5. Our existing experiments validate the effectiveness of TIDE under fixed capacity constraints. Considering that memory demands may vary with the diversity of feedback sources, we selected material tasks from 1, 2, and 3 channels and crossed them with three memory capacities B∈{40,80,160}B\in\{40,80,160\}.

Number of ChannelsB=40B=40B=80B=80B=160B=160
10.9100.9170.915†
20.8870.9130.921
30.8890.9250.923

Table: Reported Round-20 Mean Utility is the average overall utility across channels. All conditions start from an empty memory pool and use the same feedback order, retrieval count, generation budget, and evaluator, running 20 rounds of feedback updates. † The single-channel B=160B=160 condition did not reach the capacity limit within 20 rounds.

The results show that memory capacity does affect TIDE's performance, but this effect depends on the diversity of feedback sources. In single-channel tasks, B=40B=40 performs best; when the number of channels increases to 2 or 3, the smaller B=40B=40 degrades performance due to more frequent memory eviction. However, larger capacity is not always better: under the three-channel condition, B=160B=160 has about twice the number of memory tokens as B=80B=80, yet the final utility is not notably higher. Therefore, capacity should be determined based on the diversity of feedback sources and the evolution cycle; under the current setting, B=80B=80 provides a good performance–cost trade-off.


We thank the reviewer again for their attention to the rigor and practical use of our method. We noticed your interest in the recommendation domain and have answered the relevant questions. Your suggestions have led us to improve the rigor of Appendix Table 5, and have also prompted us to give good thought to the memory capacity for practical use of this method, making our solution more complete. (Note: Before the rebuttal deadline, we will 1. update Appendix Table 5, and 2. incorporate the impact of memory capacity you suggested into the appendix.) We hope we have addressed your concerns. Thank you again for your review!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.