acceptodds
Under review as a conference paper at ICLR 2027

UniMem-R1: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

Abstract

Memory managers determine what persistent agents memorize from the long and noisy context. Recent reinforcement learning (RL) approaches train such managers with downstream question answering (QA) accuracy as the reward signal, encouraging appropriate management of question-relevant memories. These methods show the QA only rewarding system can make model identify relevant memories, but their sole reliance on a finite set of sampled questions offers only a partial view of memory value. As information that is not useful for a particular QA pair can still support users' future needs, we introduce UniMem-R1, a hybrid RL framework that complements extrinsic QA supervision with an intrinsic Conditional Mutual Information (CMI) reward. From an information theory perspective, CMI measures the information gain of the memory originated from the recent dialogue after conditioning on the existing memory bank. As a result, UniMem-R1 can jointly assess the new memory's task relevance and redundancy regarding the current memory. Experiments on LoCoMo, LongMemEval, and MemoryAgentBench show that UniMem-R1 outperforms RL baselines on matched sized backbone from 8.5% to 57.0% on each dataset's overall metric.

Primary area:
reinforcement learning
Keywords:
Agentic Memory, Memory Management, Reinforcement Learning, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_c2CeOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets RL-trained memory managers for LLM agents, arguing that QA-correctness-only rewards (as in Memory-R1, MemBuilder) value a memory only through a finite set of sampled questions, leaving question-irrelevant but future-useful information unrewarded. UniMem-R1 adds an intrinsic, per-operation reward motivated by Conditional Mutual Information: the information a candidate memory fragment carries about the current dialogue session, conditioned on a locally retrieved subset of the existing memory bank. In practice this is operationalized as a residual-cosine score — context and candidate embeddings are projected off the span of retrieved memory embeddings via Tikhonov-regularized least squares, justified under a jointly Gaussian embedding model — shaped through a Gaussian "target band", mixed with session-level QA reward, and optimized with GRPO on a Qwen3-4B/8B policy over a four-partition structured memory. Auxiliary format, length, duplication, core-specificity, and attribution regularizers are applied.

Trained only on a 50-dialogue LongMemEval RL split, the method is evaluated on LongMemEval (held-out), LoCoMo (OOD), and MemoryAgentBench (OOD), against RAG, training-free prompting systems (Mem0, MemBuilder-P, an in-house Ours-P), and matched 4B RL baselines (retrained Memory-R1 and an SFT-free MemBuilder-RL). Ablations compare QA-only / CMI-only / CMI+QA and three intrinsic estimators, reward-weight and target-band ablations, training dynamics, and five qualitative case studies.

Critical strengths
  1. Cross-benchmark transfer evidence for the core claim. All RL methods are trained only on the LongMemEval split (Section 4.1, Table 6); LoCoMo and MemoryAgentBench are strictly out-of-distribution. UniMem-R1-4B beats the matched-backbone, same-training-data MemBuilder-RL-4B by +5.8 on LoCoMo, +12.3 on LME-S, and +20.0 on MemoryAgentBench overall (Tables 1–2), and even exceeds the training-free Qwen3.7-Max prompting baseline with identical memory structure (Ours-P) on LoCoMo and MAB (Table 1). This is a meaningful test of the "generalizable" claim, not just three in-distribution evaluations, and it is the right experimental design for the paper's motivating argument that QA-only rewards overfit to sampled questions.
  2. Complete implementation transparency. Full action spaces (Table 11), all prompts verbatim (Appendix I), GRPO and reward hyperparameters (Table 8), alternative estimator definitions (Appendix F), paired case studies with memory-bank statistics (Tables 12–16), and an anonymized code repository (footnote 1). This makes the work auditable and the ablations in Tables 3 and 10 interpretable, which matters because the method has many moving parts.
Critical weaknesses
  1. What is trained is not a CMI reward, so the paper's information-theoretic framing oversells a hand-tuned heuristic. The stated reward is the conditional mutual information of Eq. 2, but the implementation optimizes something quite different. First, CMI is replaced by a residual-cosine similarity score that the paper itself admits "is not calibrated in nats" (Eq. 6, p. 5). Second, this score is passed through a Gaussian target band centered at μ=0.35 (Eq. 9, p. 5), which makes the reward non-monotone: a memory with a higher estimated information gain receives a lower reward once its score exceeds 0.35. An objective that penalizes high information gain is not "maximize information gain" in any sense. Third, redundancy is in practice controlled by separate hand-crafted penalties (duplication, core-specificity, attribution weighting; Appendix D, Eq. 17) that are never ablated individually — and removing the target band itself only changes accuracy by 0.7 points (Table 10). So the experiments validate "dense similarity-based reward shaping plus hand-tuned regularizers helps," not the CMI story the paper is framed around.

  2. The headline "8.5%–57.0%" claim seems inflated, and the paper lacks variance estimates. The 57.0% endpoint (55.1 vs 35.1 on MemoryAgentBench, Tables 1–2) is measured against MemBuilder-RL — the authors' own re-implementation of MemBuilder with its SFT warm-start removed (Section 4.1); the original MemBuilder reports 84.23 on LoCoMo with the same 4B backbone, vs 68.1 for the weakened version here (Table 1). Against the stronger Memory-R1-4B baseline the gain is 30.3%, and the absolute gains over the best RL baseline are +5.8 / +8.0 / +12.8 points per dataset. More importantly, every number in the paper comes from a single training run — no seeds, standard deviations, or significance tests.

Questions for authors
  1. (Clarification) Eq. 9 makes the reward non-monotone in the estimated CMI: a fragment with residual cosine 0.9 is rewarded less than one at 0.35. What is the justification for a band-pass target on an information-gain estimate, and how were μ=0.35, σ=0.15 chosen other than by monitoring the training reward distribution? Please report sensitivity to μ and σ (Table 10 only ablates removing the band and varying α).
  2. (Clarification) Were the QA-only and CMI-only ablation runs in Table 3 trained with the same auxiliary regularizers (length, duplication, core-specificity, attribution weighting of Appendix D) as CMI+QA? If any regularizer is exclusive to the CMI+QA configuration, the Table 3 comparison is confounded; please state exactly which terms are active in each row.
  3. (Suggestion) Report mean ± std over at least 3 seeds for UniMem-R1-4B, MemBuilder-RL-4B, and the QA-only ablation on LongMemEval and MemoryAgentBench. With 50 training dialogues and 70 GRPO steps, several reported gaps (0.7–2.0 points in Tables 3 and 10) are plausibly within noise.
  4. (Clarification) The original MemBuilder uses SFT warm-start and reports 84.23 on LoCoMo with a Qwen3-4B backbone; your MemBuilder-RL reimplementation scores 68.1 (Table 1). Why is the SFT-free variant the appropriate matched baseline, and how does UniMem-R1 compare against MemBuilder with its SFT stage under your LongMemEval training split?
  5. (Clarification) Appendix G: the conditioning set is the 8 most-similar same-type entries plus 3 recent episodic entries. How sensitive is the estimator to this local-set size, given that redundancy with respect to entries outside the retrieved set is invisible to the score and handled only by the Appendix-D duplication penalties?
1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer c2Ce

We appreciate your careful review! Changes in our paper are marked in blue, and our responses are as follows:

W1, Q1:

Thank you for your insightful suggestion and question!

CMI Estimator Trade-off. For RL training efficiency, our raw score uses a fast CMI-inspired proxy to jointly measure dialogue relevance and bank non-redundancy. Inspired by [1], we use estimators to model conditional dependence, and following [2], partial correlation can be computed from regression residuals. Motivated by these works, we remove directions related to existing memories from the dialogue and candidate embeddings, then compute the residual cosine.

Gaussian Shaping Rationale & Sensitivity. The Gaussian mapping favors scores near μ\mu rather than always rewarding higher raw scores, avoiding reward hacking [3]. Since a concise fact is much shorter than a full dialogue, high scores (s>0.7s>0.7) happen when the model copies raw dialogue instead of extracting concise facts, while s→0s\to 0 indicates irrelevant text. During our experiments, we initially set μ=0.30\mu=0.30, which caused large reward swings, so we set μ=0.35\mu=0.35 to stabilize training. For σ\sigma, since raw scores lie in [0,1][0,1], choosing σ\sigma around 0.10–0.20 keeps the curve sharp enough to drop near 0 and 1, whereas a large σ\sigma (e.g., 0.8) makes it overly flat and fails to separate good and bad candidates [4]. Testing σ∈{0.10,0.15,0.20}\sigma\in\{0.10, 0.15, 0.20\} below gives similar results on LongMemEval:

σ\μ\sigma \backslash \mu0.300.350.500.70
0.1066.568.861.857.5
0.1567.068.562.557.8
0.2068.868.061.858.3

Alternative CMI Implementations. In Table 3 of our paper, logprob (67.3) and direct emb. (66.5) CMI proxies do not use residual projection, yet both clearly beat QA-only RL (56.5) on LongMemEval. Thus, gains come from the CMI objective itself rather than one specific heuristic, while residual projection keeps training fast and stable.

Experiments and clarifications about W1 and Q1 are added to Sec. 3.2–3.3, 4.3, and Appendix E (Table 13).

[1] InfoBridge: Balanced Multimodal Integration through Conditional Dependency Modeling, ICCV 25

[2] Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discovery, NeurIPS 22

[3] Scaling Laws for Reward Model Overoptimization, ICML 23

[4] Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning, ICLR 25

Q2:

Thank you for your question!

To ensure a fair comparison, all RL runs in Table 3 use the exact same Appendix D regularizers (format, length, duplication, core-specificity, and attribution weighting), except the Gaussian shaping (Only for CMI). Notably, even with all duplication and length penalties active, QA-only gets 56.5 vs. 68.8 for CMI+QA, showing hand-crafted penalties alone cannot replace CMI. We have further clarified this in Sec. 4.3.

W2,Q3,Q4:

Thank you for your suggestion and questions!

Matched Baseline & Headline Claim. To study RL reward design without mixing in teacher distillation gains, we train all RL methods without SFT on the exact same backbone and split. Also, original MemBuilder's Claude 4.5 Sonnet SFT trajectories are not released in its public repo, so SFT-free MemBuilder-RL is the cleanest matched baseline. We also removed "8.5%–57.0%" in the Abstract and Intro, and clarified this baseline in Sec. 4.1.

Multi-seed Runs. Due to time and resource limits, we could not finish 3-seed runs in time, and will add full multi-seed results (mean ± std) in the camera-ready version.

Q5:

Thank you for your question!

Following your question, we tested local-set sensitivity by replaying the LongMemEval training trajectory, keeping each session, candidate, and pre-operation bank fixed while varying top-kk similar same-type entries and rr recent episodic entries. Compared against the full bank (k=∞,r=∞k=\infty, r=\infty), our default (k=8,r=3k=8, r=3) stays within ~5% deviation, showing low sensitivity to local-set size.

In the first table, each cell (left / right) shows raw-score deviation (%) / Gaussian-shaped reward deviation (%) (std of score differences vs. the full bank):

k\rk \backslash r036
46.5% / 12.3%7.3% / 3.7%6.3% / 5.6%
87.7% / 5.3%5.5% / 4.3%5.4% / 3.5%
167.6% / 6.0%5.8% / 4.4%5.1% / 3.6%

In the second table, each cell (left / right) shows scoring time per edit (ms) / relative time change (%) vs. the full bank:

k\rk \backslash r036
40.2 / -85%0.2 / -83%0.3 / -80%
80.3 / -81%0.3 / -79%0.3 / -77%
160.4 / -73%0.4 / -71%0.4 / -69%

Although k=16,r=6k=16, r=6 has <1% lower deviation, our setting (k=8,r=3k=8, r=3) saves 10% more time per edit, which matters across numerous RL actions. Experiments and clarifications are added to Sec. 3.2 and Appendix G (Tables 14–15).

Reviewer_smveOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniMem-R1, a reinforcement learning framework for long-term memory management. Conventional methods train memory managers using only question-answering accuracy, potentially overlooking information that is irrelevant to current questions but valuable in the future. UniMem-R1 combines a QA reward with a conditional mutual information reward: the former ensures downstream utility, while the latter evaluates whether a candidate memory is relevant to the current dialogue and non-redundant with existing memories. The system manages core, episodic, semantic, and procedural memories and learns operations such as adding, updating, merging, and skipping through GRPO. Experiments show that UniMem-R1 outperforms matched RL baselines across three long-term memory benchmarks and generalizes better across tasks.

Critical strengths
  1. Comprehensive reward design: UniMem-R1 combines QA and CMI rewards, encouraging the model to retain information that is both useful for downstream tasks and novel relative to existing memories.
  2. Fine-grained training signals: The CMI reward provides dense feedback for individual memory operations, such as adding, updating, and merging, improving credit assignment in long-horizon reinforcement learning.
  3. Strong cross-task generalization: Although trained only on LongMemEval, UniMem-R1 outperforms matched RL baselines on out-of-distribution benchmarks such as LoCoMo and MemoryAgentBench, supported by extensive ablation studies.
Critical weaknesses

The novelty is moderate rather than fundamental. The work primarily integrates established structured-memory management, GRPO, QA-based supervision, and information-theoretic rewards. Its main original contribution lies in adapting conditional mutual information to operation-level memory valuation and introducing a residual-projection estimator, rather than proposing a fundamentally new memory architecture or reinforcement learning algorithm.

Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer smve

Thank you for your review!

In this paper, we identify a key limitation of QA-only rewards: finite sampled questions leave potentially useful information unsupervised. Based on this novel observation, we first propose a simple yet effective method named UniMem-R1. This method combines a CMI-motivated signal with standard QA supervision, enabling the agent to learn not only what answers questions correctly, but also what information is informative for future use. Overall, this work enhances the memory management capabilities of RL-based dialogue agents and offers a new perspective on complementing QA supervision with conditional-information signals. We hope the proposed framework provides more insights for future research in both memory-augmented reasoning and information selection. We have incorporated this discussion into the conclusion (marked in blue) to help readers better understand the scope and motivation of our contribution.

Dear Reviewer smve,

Thank you very much for your careful review!

As the discussion period ends, we look forward to your feedback to confirm whether our responses have adequately addressed your concerns. Your feedback will help us improve the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

Authors

Dear reviewer smve,

Thanks for your time and effort in reviewing ICLR2027 submissions. The authors have submitted their responses to your review. Please take the time to read, raise any further comments, and discuss them with the authors. Thanks very much!

AC

Reviewer_KcWYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a finite set of QA rewards adequately evaluates long-term memory. A fact absent from training questions may still matter for future requests, motivating a query-independent memory score alongside session-level QA feedback. A shared policy edits Core, Episodic, Semantic, and Procedural memory. For each candidate, the scorer retrieves local entries of the same type, uses frozen embeddings and regularized linear projection to residualize the dialogue and candidate against existing memory, and computes residual cosine similarity as a proxy for information added relative to the bank. Add and merge operations use a nonnegative score; replacement and update use the difference between new and old fragment scores.

A Gaussian target band centered at 0.35 with width 0.15 shapes the scores. Averaged operation rewards enter the session reward with weight 0.3 alongside QA correctness. Training retains format, length, duplication, and attribution regularizers and uses session-level GRPO. Optional retrieval-indexed memory, RM, restores source dialogue associated with retrieved entries. The primary Qwen3-4B experiment trains on synthetic session QA from 50 LongMemEval conversations, tests on 400 held-out questions, and evaluates transfer to LoCoMo and MemoryAgentBench. Table 3 reports 56.5 for QA-only, 45.0 for intrinsic-only, and 68.8 for their combination without RM; the combined configuration with RM scores 68.5. Other scorers, reward weights, and some 8B results are also evaluated.

Critical strengths
  1. Conditioning candidate scores on both the current dialogue and existing memory can provide feedback for content absent from sampled QA, while avoiding a novelty-only objective that rewards unrelated material. The main scorer uses frozen embeddings and a small linear solve, without generating and answering another auxiliary question for every candidate. It is a practical additional supervision signal.

  2. Table 3 reports 56.5 for QA-only and 68.8 for the combination on the same backbone and answering pipeline without RM, supporting a substantial benefit from additional supervision. Intrinsic-only scores 45.0, showing the need for task feedback. RM configurations, direct-embedding and logprob scorers, target-band shaping, and reward weights are evaluated. These experiments help assess the roles of intrinsic feedback, retrieval structure, and reward shaping.

Critical weaknesses
  1. Pages 4 and 5 motivate residual cosine through partial correlation for scalar jointly Gaussian variables, without establishing how this statistical relation applies to a single pair of text embeddings or validating the required information ordering on actual memory edits. The score is explicitly uncalibrated in nats, and the Gaussian band is disclosed and ablated as an empirical regularizer. However, direct discrimination of relevant new facts, duplicates, and irrelevant content is not validated. In Table 3 with RM, residual scoring reaches 68.5 versus 66.5 for direct embeddings, without reported variation across training runs or an interval for the difference. This supports a useful scorer but not yet its CMI interpretation or a robust advantage over simpler scoring.
  2. LoCoMo and MemoryAgentBench lack reward ablations corresponding to Table 3, so the complete system’s transfer gains cannot be assigned specifically to the new reward. Page 7 acknowledges that RM restores source examples and labels that compressed entries can omit, particularly helping MCC. The gain without RM on LongMemEval rules out that explanation there, but does not replace transfer comparisons with retrieval and regularizers fixed. Table 4 and Appendix J provide helpful examples, without quantifying whether improvements concern facts absent from training QA.
  3. Table 2 uses modified MemoryAgentBench scoring and excludes FC-MH. Appendix K explains these choices, and methods share the answer model and corresponding scoring, supporting comparisons within the stated protocol. However, agreement with official scoring or independent annotations is not reported for the new LLM judging of tasks such as MCC or the ROUGE-L summary evaluation. A reader overriding counterfactual memory also does not by itself establish that FC-MH cannot compare memory systems: following updated stored information is part of the relevant evaluation goal. Without scoring the same outputs under both protocols and retaining full subset coverage, the scope of the overall gain and selective-forgetting conclusions remains unclear.
Questions for authors
  1. Can controlled new facts, duplicates, and irrelevant fragments test the ordering of raw scores and shaped rewards, including repeating the test once the same candidate is already stored? Does the 2.0-point residual versus direct-embedding difference persist across training runs or paired intervals?

  2. What are the QA-only and combined-reward transfer results on LoCoMo and MemoryAgentBench with RM, retrieval, and other regularizers fixed? Can error and memory comparisons quantify gains on facts absent from training QA?

  3. How do Table 2 rankings change under official scoring of the same predictions, MCC accuracy after extracting class labels, and inclusion of FC-MH? Does an oracle-memory control support the explanation that FC-MH is primarily limited by the reader rather than memory construction?

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer KcWY

W1, Q1:

Thank you for your suggestion!

CMI Scoring & Empty-Bank Rule. Our CMI reward directly measures the conditional mutual information between a candidate memory and the current dialogue given the memory bank. When the bank is empty, there is nothing to condition on, so we skip projection and use s=max⁡(0,cos⁡(eC,em))s=\max(0,\cos(e_C,e_m)).

Purpose of Gaussian Shaping. For nonzero ss, we use Gaussian shaping g(s)=exp⁡[−(s−0.35)2/(2⋅0.152)],g(0)=0g(s)=\exp[-(s-0.35)^2/(2\cdot0.15^2)], g(0)=0 for the following purposes:

  1. Stabilize RL training via reward smoothing [1,2,3], especially during cold start before the bank is established (e.g., g(0.05)=0.135,g(0.35)=1,g(0.80)=0.011g(0.05)=0.135, g(0.35)=1, g(0.80)=0.011). Thus, Gaussian shaping does not keep raw-score ordering.
  2. Avoid reward hacking [4]. A very high score (s>0.8s>0.8) happens when the model copies the original dialogue instead of extracting concise facts, while s→0s\to 0 indicates irrelevant text.

Controlled Discrimination & Ordering Validation. To test score ordering without QA interference, we run 160 controlled trials on 20 Qwen3.7-Max generated profiles, scoring candidates before → after storing A's paraphrase (Exact Repeat scores B before → after storing B):

  1. Duplicate: Score A, store A's paraphrase, score A again.
  2. New Fact: Score B, store A's paraphrase, score B again (B stays unstored).
  3. Irrelevant: Score D (absent from dialogue) before and after storing A's paraphrase.
  4. Exact Repeat: After storing A's paraphrase, score B, store B, score B again.
New FactDuplicateIrrelevantExact Repeat
Raw score0.40→0.33 (-16.6%)0.40→0.18 (-56.2%)0.18→0.10 (-46.6%)0.33→4.4e-5 (-100%)
Shaped reward0.79→0.78 (-1.6%)0.79→0.52 (-34.3%)0.53→0.29 (-45.9%)0.78→0.00 (-100%)
Direct Emb.0.20→0.19 (-5.3%)0.20→0.03 (-82.8%)0.18→0.17 (-6.2%)0.19→1.4e-8 (-100%)

Our scorer preserves New Fact > Duplicate > Irrelevant > Exact Repeat. Under 95% paired bootstrap intervals on New Fact > Irrelevant, shaped residual reaches 89.4% vs. 72.5% for shaped Direct Emb. (paired gain 16.9% [+8.1%, +25.6%]; raw 98.8% vs. 72.5%). We have added the Gaussian shaping rationale to Sec. 3.3 and the controlled discrimination experiments to Sec. 4.3 (Table 4).

W2, Q2:

Thank you for raising this point!

To directly test whether our transfer gains come from the CMI reward itself, and whether CMI retains useful facts not covered by training QA, we compare +QA and +CMI+QA on LoCoMo and MemoryAgentBench (MAB) while keeping the training backbone, pipeline, and regularizers identical across both runs. Since both models are trained only on LongMemEval (LME), LoCoMo and MAB can evaluate Out-Of-Distribution (OOD) query types never seen during training:

LoCoMoLMEMAB
+QA63.556.546.7
+CMI+QA73.968.855.1

With fixed retrieval and regularizers, adding CMI improves LoCoMo (+10.4) and MAB (+8.4). These gains show CMI retains useful facts missed by training QA.

We have added these OOD transfer experiments and analysis to Sec. 4.3 and Appendix E (Table 11).

W3, Q3:

Thank you for raising this point!

Evaluation Protocol. We documented our metric rationale in Appendix K.3. We keep most official task-native metrics where suitable, replacing Substring Exact Match (SEM) with LLM-as-Judge on free-form QA/MCC, as SEM misses paraphrases and rewards verbose outputs that hit substrings by chance or quote both old and updated facts.

Oracle-Memory Check on FC-MH. To verify FC-MH is limited by the reader rather than memory construction, we test two readers plus an Oracle Memory upper bound that injects gold source evidence directly into the reader's input (% under official Substring Exact Match):

MethodQwen3.7-MaxDSV4-Flash
RAG (top-5/10)1.0 / 2.04.0 / 4.0
Ours-P6.05.0
MemBuilder-P4.02.0
Mem04.02.0
Memory-R1-4B3.04.0
MemBuilder-RL-4B5.03.0
UniMem-R1-4B5.03.0
Oracle Memory7.07.0

All managers score 1.0%–6.0%, and even Oracle Memory gets only 7.0% on both LLMs, where managers faithfully store conflicting updates across turns, and reading them together across hops triggers reader reasoning that spots the counterfactual setup and "corrects" it via world knowledge [5] (true in reality, but scored wrong).

We have added these experiments and analysis to Sec. 4.3 and Appendix K.2 (Table 22).

[1]. Variance Aware Reward Smoothing for Deep Reinforcement Learning, Neurocomputing 21

[2]. DreamSmooth: Improving Model-Based Reinforcement Learning via Reward Smoothing, ICLR 24

[3]. Human-level control through deep reinforcement learning, Nature 2015

[4]. Scaling Laws for Reward Model Overoptimization, ICML 23

[5]. Adaptive chameleon or stubborn sloth, ICLR 24

Dear Reviewer KcWY,

Thank you very much for your careful review!

As the discussion period ends, we look forward to your feedback to confirm whether our responses have adequately addressed your concerns. Your feedback will help us improve the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

Authors

Dear reviewer KcWY,

Thanks for your time and effort in reviewing ICLR2027 submissions. The authors have submitted their responses to your review. Please take the time to read, raise any further comments, and discuss them with the authors. Thanks very much!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.