acceptodds
Under review as a conference paper at ICLR 2027

RegimeCouncil: Paired-Evidence Routing for Memory Conflict Governance in LLM Agents

Abstract

Language agents increasingly rely on long-term memory to sustain extended interactions with their users, yet stored facts become stale as the world changes, and these agents often learn that a fact was wrong only through delayed feedback, after already acting on that fact. A central question is how far a single correction should propagate: should it govern only later uses of the same fact, or also how much the agent trusts memory on other facts? Existing systems typically adopt one of these two policies at design time. We formalize this choice as a sequential decision problem—memory conflict governance—in which memory validity is hidden at decision time and outcomes are revealed with delay, and prove that neither policy is uniformly sufficient. We then propose RegimeCouncil, which retains both policies and selects between them using delayed paired outcomes, without retraining or assuming a fixed dominance order. Two advisors score the advantage of using memory over a no-memory anchor: Global pools evidence across stored facts, while Local uses only the queried fact. A Meta Governor defaults to Local and switches to Global only when Global sustains a positive recent margin; under delayed feedback, this switch is provably reached within a bounded number of decisions. On delayed-feedback streams, RegimeCouncil outperforms the better fixed-scale baseline by – points on disagreement episodes. The same frozen router transfers to naturally occurring drift on TempLAMA and TAQA. Ablations confirm that every component is important: removing each of Global, Local, and the Meta Governor lowers accuracy by – points.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM agents, agent memory, memory conflict governance, delayed feedback

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_XtwaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies a problem regarding long-term memory in LLM agents: whether delayed feedback about stale memories should influence only the queried fact or memories use across all facts. In order to address this problem, this paper formalizes a sequential decision problem and demonstrates that the strategies adopted in previous work are not sufficient enough. Besides, this paper further proposes a RegimeCouncil method that combines a globally pooled advisor and a per-key advisor. As validation, both theoretical and empirical results are provided in the paper.

Critical strengths
  • This paper introduces local and global feedback, and the distinction between the two types of feedback is clear and relevant.

  • The router proposed in the paper is easy to implement, interpretable, and computationally lightweight.

Critical weaknesses
  • The formulation of the research problem in this paper is poor. In the Introduction section, this paper fails to state the problem well and clearly formulate their research problem. Too many examples are mentioned, which makes me confused and makes them feel trivial about their work.

  • The statements of previous work and the definitions in the paper are quite confusing. After reading the content, I can hardly understand the core problem and technical insights here.

  • The writing of this paper is quite poor. The sentences and structure of this paper do not fit the norms of a research paper.

Questions for authors
  1. Could you please provide a clear summary regarding your research problem in this paper, including the memory conflict governance and the concrete problem regarding this paper?

  2. Why did you mention "We index each stored fact by a key ..." (cf. Lines 52-73)? Is this a method proposed in your work? This part is quite confusing.

  3. What are insights regarding the previous work mentioned in Lines 75-80? Are these terms defined by you or common sense in the community?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Xtwa (Part1, Core Contribution)

Thank you very much for the constructive review and for the helpful suggestions. We address all points in three comments. Before answering each of your questions and weaknesses point by point, we want to highlight why the underlying contribution of this work is both practically important and new. This work's core contribution is solving an unaddressed practical problem in agent memory with strong empirical gains.

  • A critical real-world problem that existing agent memory systems leave open: In deployed LLM assistants, stored user facts (e.g., city, employer, preferences) quietly go stale over time, causing severe accuracy drops (e.g., a >20-point drop on knowledge-update queries in LongMemEval [Wu et al., 2025]). Existing memory frameworks (MemGPT, Mem0, Zep, Mem1, MemRL) focus on what to write, retrieve, or compress, or how to overwrite a single item once its replacement value is given. However, they do not address how an agent should govern memory use before the memory store is updated (e.g., under asynchronous batched updates or when negative feedback flags an error without supplying the new fact): once delayed feedback reveals that using memory hurt on one fact, should that signal suppress memory use only on that specific fact (local scope), or across the memory store (global scope)?
  • Theoretical and empirical validation: We prove that neither fixed scope is sufficient (Proposition 2: global pooling fails under asynchronous per-fact drift, while key-local isolation fails on sparse/unseen keys and store-wide shifts), and propose RegimeCouncil to arbitrate between them online at negligible compute cost (2.29 µs/decision, 0 extra user-facing LLM calls). Across 30 held-out evaluation settings (33,792 episodes) spanning LongMemEval, TempLAMA, and TAQA on Qwen3-30B-A3B and GPT-4o, RegimeCouncil outperforms the best single advisor in all 30 of 30 settings (+7.3 points median disagreement-slice gain; +4.5 points full-stream gain over Global), reduces harmful memory exposures by 69% (from 2,273 to 707 episodes), and improves accuracy by +27.9 points under clustered shifts.
AuthorsOpenReview ↗

Response to Reviewer Xtwa (Part2, Q1 & W1 and Q2 & W2)

Q1 & W1. Clear summary of the research problem and memory conflict governance (and removing distracting examples)

  • Why Section 1 felt cluttered (Addressing W1): In the submission, lines 45–60 mixed the main Beijing → Shanghai example with several briefly mentioned attributes (volleyball_record, job, hobby) before stating the formal problem. In the revision, we removed those extra examples and immediately follow the single Beijing → Shanghai example with the formal problem definition below:
  • Concrete problem formulation:
    1. Setting: An LLM agent maintains a long-term memory store of KK tracked facts/attributes indexed by k∈{1,…,K}k \in \lbrace 1,\dots,K \rbrace (e.g., user_city). At step tt, a query arrives for fact ktk_t with prompt context ztz_t. The agent chooses a binary action at∈{0,1}a_t \in \lbrace 0,1 \rbrace: inject the stored memory record for ktk_t (at=1a_t{=}1) or answer without it using the base model, which we call the no-memory anchor (at=0a_t{=}0).
    2. Why it is non-trivial: Because user facts change without explicit deletion, the stored record for ktk_t may be valid or stale, and the prompt ztz_t alone does not reveal which is true (decision-time unobservability, Assumption 1). Only after a delay of up to τ\tau steps—when a user correction, complaint, or task outcome arrives—can the agent evaluate whether using memory helped or hurt on episode tt, yielding the paired utility difference dt=ut,mem−ut,0∈[−1,1]d_t = u_{t,\text{mem}} - u_{t,0} \in [-1,1] (Assumptions 2–3).
    3. The research problem (memory conflict governance): Once delayed outcomes did_i arrive, at what scope should they govern future memory-use decisions ata_t?
      • A key-local policy uses only past outcomes from the same fact (ki=ktk_i = k_t).
      • A globally pooled policy pools recent outcomes across all facts (ki∈{1,…,K}k_i \in \lbrace 1,\dots,K \rbrace).
  • Core technical insights:
    • Proposition 1: Any stationary policy using only the current prompt ztz_t suffers a constant regret floor δ=pqΔ>0\delta = pq\Delta > 0, whereas conditioning on delayed outcome history escapes this floor with regret O((S+K)m/T)O((S+K)m/T).
    • Proposition 2: Neither fixed scope is uniformly sufficient—global pooling suffers Ω(T)\Omega(T) regret when facts drift asynchronously (majority keys overwrite minority keys), while key-local isolation suffers Ω(T)\Omega(T) regret on unseen or sparse keys (zero local history while pooled evidence is already informative).
    • RegimeCouncil & Proposition 3: RegimeCouncil maintains both the Global Advisor (WgW_g) and the Local Advisor (WℓW_\ell), and uses a Meta Governor that defaults to Local and switches to Global only when Global accumulates a strictly positive recent counterfactual surplus At>0A_t > 0 over the last WmW_m revealed episodes (guaranteeing finite-time switching within Wm+τW_m+\tau steps).

Q2 & W2. Why we wrote "We index each stored fact by a key …" (Lines 52–73), and whether it is a proposed method

  • Direct answer: It is not a proposed method; it is standard key–value notation so that each tracked attribute has an identifier k∈{1,…,K}k \in \lbrace 1,\dots,K \rbrace, allowing us to distinguish "feedback on the same fact" (ki=ktk_i = k_t, Local) from "feedback pooled across all facts" (ki∈{1,…,K}k_i \in \lbrace 1,\dots,K \rbrace, Global).
  • Why Lines 52–73 were confusing and how we fixed them: In the submission, we placed the sentence "We index each stored fact by a key..." in Paragraph 2 of the Introduction right after critiquing prior work, which followed the rhetorical pattern of a methodological claim and made a simple notation look like a proposed contribution. In the revision, we deleted this sentence from the Introduction and introduce k∈{1,…,K}k \in \lbrace 1,\dots,K \rbrace once in Section 3.1 inside the formal problem setup block.
AuthorsOpenReview ↗

Response to Reviewer Xtwa (Part3, Q3 & W2 and W3)

Q3 & W2. Insights on previous work (Lines 75–80), and whether "global policy" / "local policy" are our terms or community terms

  • Are these terms defined by us or community terms? Global policy, local policy, and memory conflict governance are terms defined by us in this paper to formalize two feedback-sharing scopes that prior systems adopt implicitly. In the submission, introducing them in parentheses in Lines 75–80 without an explicit definition made it unclear whether they were established community terms. In Section 1 (Paragraph 2) of the revised PDF, we now state explicitly: "When adapting memory use from past outcomes, however, prior designs implicitly fix one of two opposite sharing scopes, which we define as a local policy and a global policy."
  • Concrete insights on previous work (Lines 75–80): Rather than stating abstractly that prior systems fix one policy at design time, the revised Introduction and Section 2 explicitly map prior families to these two scopes and explain their failure modes:
    1. Implicit global-scope mechanisms (e.g., global retrieval-confidence gates or shared recency/trust windows in RAG and memory pipelines) tune or gate memory injection across all queries jointly—which fails when a few volatile attributes (e.g., current travel city) go stale and cause the gate to suppress valid stable memories (e.g., dietary restrictions).
    2. Implicit key-local mechanisms (e.g., per-item utility/Q-value updates in MemRL [Zhang et al., 2026], or per-fact overwrite in Mem0 [Chhikara et al., 2025] and Zep [Rasmussen et al., 2025]) update only the accessed memory record in isolation—which cannot transfer staleness warnings across keys when a batch of facts goes stale together (e.g., during batched consolidation after a relocation or job change) or when querying a newly added key with no prior history.

W3. Rewriting sentences and structure to fit research-paper norms

To address W3 ("The sentences and structure of this paper do not fit the norms of a research paper"), we rewrote the Abstract and Sections 1–5 in the uploaded PDF:

  1. Section 1 (Introduction): Rewritten into four standard paragraphs: (i) Problem motivation via the single Beijing → Shanghai example, (ii) Explicit definitions of local policy vs. global policy and how prior memory systems map onto them, leading to memory conflict governance, (iii) Clear overview of RegimeCouncil and Propositions 1–3, and (iv) Three-point summary of contributions.
  2. Section 2 (Related Work): Reorganized into three thematic paragraphs, each ending with an explicit contrast between prior memory/bandit/expert-tracking methods and our online feedback-scope arbitration.
  3. Section 3 (Method): Section 3.1 now defines all problem notation (kt,zt,rt,at,ut,Ftk_t, z_t, r_t, a_t, u_t, \mathcal{F}_t) and Assumptions 1–3 in one self-contained block before Proposition 1, followed cleanly by paired evidence dsd_s (Section 3.2), the Global and Local Advisors (Sections 3.3–3.4, Proposition 2), and the Meta Governor AtA_t (Sections 3.5–3.6, Proposition 3).
Reviewer_m7xeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper handle long-term memory issue in LLM agents. A central question is how far a single correction should propagate. Authors propose RegimeCouncil to retain both policy (local and global) then, let them to select which to use. They report the need of all components (local, global, meta governor) and show how RegimeCouncil outperforms the baseline performance in long-term memory tasks.

Critical strengths
  1. This paper addresses a timely problem in LLM agents, as long-term memory is becoming more common because of serving as personal agents.
  2. The experiments are conducted under well-controlled setting.
Critical weaknesses
  1. RegimeCouncil rely on an idealized experiment environment.
    • RegimeCouncil basically depends on the Assumption3 (replayable evaluation) which require knowing the outcomes of both using memory and not using it. However, real-world feedback (complaint, a failed task) only show the outcome of executed action.
    • In this paper, experiments are conducted in a synthetic environment with cached answer and gold labels, so the outcomes of all episodes are known immediately. Settings where feedback is sparse or given only for the executed action are not tested.
  2. Necessity of Meta Governor
    • According to the paper, most disagreements between local and global occur when the local has no signal. In this case a simple rule could replace the Meta Governor (i.e., following the global when there is no signal from local). A comparison with this rule will justify the need of Meta Governor but this paper lacks that part.
Questions for authors
  1. Could you justify your choice of conducted environment setting which is quite different from the reality?
  2. Could you show the outcome that I commented in the Weakness 2?
6 comments
AuthorsOpenReview ↗

Response to Reviewer m7xe (Part1, W1 & Q1)

We deeply appreciate Reviewer m7xe's constructive and insightful comments. We address all points in three comments.

W1 & Q1. Justifying the controlled setup (Q1), relaxing Assumption 3 when feedback only shows the executed action (W1.1), and testing sparse / delayed / executed-only feedback (W1.2)

1. Response to Q1 — Justifying the controlled benchmark setup:

  • Real data, not synthetic toy tasks: All questions and evolving facts come directly from three public benchmarks—LongMemEval (knowledge-update user sessions), TempLAMA (120 time-varying Wikidata facts), and TAQA (100 evolving Wikipedia tables)—using real Qwen3-30B-A3B and GPT-4o completions, and Table 4 (Section 4.2) directly evaluates the real historical switch chronologies of TempLAMA and TAQA.
  • Why timestamps are stripped and completions cached: Raw benchmarks include explicit dates in the prompt, allowing a reader to pick the latest fact from ztz_t alone without testing hidden staleness (Assumption 1). Stripping temporal cues models the realistic case where a user's city or preference changes silently without an explicit delete. Caching the completions per item ensures every baseline sees identical LLM outputs, isolating routing performance from generation variance.

2. Response to W1 (Point 1) — How RegimeCouncil works when real-world feedback (a complaint or failed task) only shows the outcome of the executed action: When the agent only observes whether the executed action ata_t succeeded (ut,at+1u_{t,a_t}{+}1) or triggered a user complaint / failed task (ut,at−1u_{t,a_t}{-}1), RegimeCouncil can operate in two ways without benchmark gold labels:

  • Mechanism A — Pure executed-action-only bandit (relaxing Assumption 3 completely, no unexecuted action evaluated): The agent observes only the outcome ut,atu_{t,a_t} of the action actually shown to the user, and forms an unbiased inverse-propensity estimate d^t\hat d_t (with ε=0.05\varepsilon{=}0.05 exploration so E[d^t]=dt\mathbb{E}[\hat d_t] = d_t) to drive the Global Advisor, Local Advisor, and Meta Governor directly—with zero extra model calls or counterfactual scoring.
  • Mechanism B — Asynchronous check against the same complaint (0 ms user latency, no gold labels): When a user complaint arrives on a memory-based answer (e.g., "I don't live in Beijing anymore"), the agent can check a background no-memory completion against that same complaint (whether the fallback avoided assuming Beijing) to obtain ut,0u_{t,0} and recover paired dtd_t without needing ground-truth answers.

3. Response to W1 (Point 2) — New experiments on sparse feedback, non-immediate (delayed) outcomes, and executed-action-only feedback: We tested all three conditions across all 30 held-out settings (33,792 episodes, keeping the paper's fixed Wg=32,Wℓ=1,Wm=8W_g{=}32, W_\ell{=}1, W_m{=}8, gate 00):

Realistic feedback condition (across all 30 held-out settings)Submitted RegimeCouncilComparison with baselines
Executed-action-only feedback (Mechanism A: only the shown answer receives pass/complaint feedback; no u0u_0 call)68.9% mean full-stream accuracy (31.3 harmful episodes per setting)Outperforms Global (63.2% accuracy, winning in all 30 of 30 settings and reducing harmful episodes by half from 60.6 to 31.3), Follow-recent-winner (68.5% accuracy, winning in 23 of 30 settings, two-sided sign test p=0.002p{=}0.002), and Local (68.5% accuracy, winning in 18 of 30 settings)
Complaint-checked paired feedback (Mechanism B: background no-memory answer checked against the same complaint; no gold labels)71.6% mean full-stream accuracy (+7.3 points median disagreement-slice gain over max⁡(G,L)\max(G,L))Outperforms Global (67.1% accuracy) and Local (68.4% accuracy) in all 30 of 30 settings, with 0 ms added user-facing latency
Sparse feedback (outcomes revealed on only 75% or 50% of episodes, and missing on the rest; 5 seeds × 30 settings = 150 runs)+5.7 points (at 75%) and +3.2 points (at 50%) median disagreement-slice gain over max⁡(G,L)\max(G,L), achieving positive gains in 149/150 and 130/150 runsEvent-indexed windows ignore unrevealed turns; outperforms Follow-recent-winner and Fixed Share (+5.5 and +2.9 points median gain)
Non-immediate (delayed) outcomes (random delay τt∼Uniform{1..8}\tau_t \sim \mathrm{Uniform}\lbrace 1..8\rbrace or {1..16}\lbrace 1..16\rbrace steps; 5 seeds × 30 settings = 150 runs)+6.7 points (at {1..8}\lbrace 1..8\rbrace) and +5.9 points (at {1..16}\lbrace 1..16\rbrace) median disagreement-slice gain over max⁡(G,L)\max(G,L), achieving positive gains in 149/150 and 140/150 runsOutperforms Fixed Share and Follow-recent-winner (which suffer negative gains in 17 of 150 runs under Uniform{1..16}\mathrm{Uniform}\lbrace 1..16\rbrace)
AuthorsOpenReview ↗

Response to Reviewer m7xe (Part2, W2 & Q2)

W2 & Q2. Results of "following Global when there is no signal from Local" and necessity of the Meta Governor

Across all 30 held-out streams (33,792 episodes), advisor disagreements (gt≠ℓtg_t \neq \ell_t) fall into two distinct cases:

  • Case 1 — When there is no signal from Local (the queried key is unseen or its latest outcome was a tie, d=0d{=}0; 81% of disagreements, 10,451 episodes): this is the exact case targeted by your W2 rule ("following the global when there is no signal from local").
  • Case 2 — When Local does have a non-zero signal (d≠0d \neq 0) that conflicts with Global (19% of disagreements, and the dominant case whenever a cluster of facts becomes stale together, Figure 1 scene 2): here Local has a signal, so a rule that only specifies what to do when Local has no signal cannot arbitrate whether Local's signal is still valid or has gone stale.

Below we compare your W2 simple rule (follow Global when Local has no signal, and keep the Local default otherwise, replacing the Meta Governor) directly against the submitted RegimeCouncil (which uses the Meta Governor At>0A_t>0):

Evaluation setting & disagreement regimeMetricYour W2 Simple Rule (follow Global when Local has no signal; no Meta Governor)Submitted RegimeCouncil (with Meta Governor At>0A_t>0)
Case 1 — 30 held-out streams (33,792 episodes; 81% of disagreements have no Local signal, d=0d{=}0)Mean full-stream accuracy & median disagreement-slice gain over max⁡(G,L)\max(G,L)72.4% full-stream accuracy (+9.7 points median disagreement-slice gain)71.6% full-stream accuracy (+7.3 points median disagreement-slice gain)
Case 1 — 30 held-out streams (safety / downside risk)Total harmful memory exposures across 33,792 episodes752 harmful episodes (2.23% of all episodes)707 harmful episodes (2.09% of all episodes — lowest of all adaptive routers)
Case 2 — Clustered corrections (sync_multi, 54-key pool; Local has a stale non-zero signal d=+1d{=}+1)Disagreement-slice accuracy66.3%94.2% (+27.9 percentage points higher than Simple Rule)
Case 2 — Single store-wide shift (sync_switch, 54-key pool; Local has a stale non-zero signal d=+1d{=}+1)Disagreement-slice accuracy & total harmful episodes91.3% accuracy, with 91 harmful episodes93.5% accuracy, with only 37 harmful episodes (a 59% reduction in harmful episodes)
Case 1 + Case 2 — Two-stage stream (asynchronous flips in the first half, followed by store-wide flips in the second half)Disagreement-slice accuracy & total harmful episodes69.7% accuracy, with 102 harmful episodes68.2% accuracy, with only 75 harmful episodes (a 26% reduction in harmful episodes)

What these results show:

  1. In Case 1 (when Local has no signal, d=0d{=}0), your simple rule is highly effective—trading a small increase in harmful episodes for higher accuracy: Because the submitted Local Advisor abstains (at=0a_t{=}0) on d=0d{=}0 to guarantee zero harmful memory exposure, the submitted RegimeCouncil achieves the lowest harmful exposure of all methods (707 episodes). Your simple rule instead follows Global on d=0d{=}0, incurring 45 additional harmful episodes across 33,792 decisions (752 vs. 707, i.e., 2.23% vs. 2.09% of episodes) in exchange for a +0.8-point full-stream (+2.4-point slice) accuracy gain.
  2. In Case 2 (why the Meta Governor is still necessary): When a cluster of facts changes together (sync_multi, sync_switch), each key's latest outcome is non-zero (d=+1d{=}+1) but stale. Because Local has a signal, the rule "follow Global when there is no signal from Local" does not trigger, leaving the agent locked onto stale per-key signals (66.3% accuracy and 91 harmful episodes). The Meta Governor (At>0A_t>0) is needed precisely here: it detects that Global is outscoring Local on recent revealed episodes and overrides the stale Local signals (reaching 94.2% accuracy, a +27.9-point improvement, and reducing harmful episodes from 91 to 37, a 59% reduction).
  3. Thank you—combining your W2 rule (for Case 1) with the Meta Governor (for Case 2) improves the paper: Using your rule when Local has no signal (d=0d{=}0) while retaining the Meta Governor (At>0A_t>0) when Local has a signal achieves both high accuracy on the 30 held-out streams (72.3% full-stream accuracy, +9.8 slice gap, and 748 harmful episodes) and strong robustness on clustered updates (94.2% accuracy while reducing harmful episodes from 91 to 40, compared with 91 harmful episodes without the Meta Governor). We have added this exact comparison to Section 4.3 and Appendix E of the revised PDF and credited your review.
AuthorsOpenReview ↗

Response to Reviewer m7xe (Part3, Improving Presentation and Readability)

Improving Presentation and Readability (Addressing Presentation Score: 1)

We took your readability rating seriously and made three concrete structural revisions across the manuscript to make the paper self-contained and easy to follow on a first read:

  1. Clean four-paragraph Introduction with a single running example (Section 1): Removed the distracting secondary attribute examples (volleyball_record, job, hobby) and misplaced key-indexing sentence from Section 1, keeping only the Beijing → Shanghai running example followed immediately by a concise formal overview of memory conflict governance, explicitly defined local vs. global baseline policies, and a clear bulleted contribution summary.
  2. Explicit scope comparisons in Related Work (Section 2): Reorganized Section 2 so that each paragraph explicitly contrasts prior memory write/retrieval mechanisms (e.g., global confidence gates vs. per-item updates in MemRL and Mem0) with our online feedback-scope arbitration.
  3. Self-contained problem setup in Section 3.1 and structured protocol walkthrough in Section 4: Defined all problem notation (kt,zt,rt,at,ut,Ftk_t, z_t, r_t, a_t, u_t, \mathcal{F}_t) and Assumptions 1–3 cleanly in Section 3.1 before Proposition 1 (followed step by step by paired evidence dsd_s, Global Advisor gtg_t, Local Advisor ℓt\ell_t, and Meta Governor AtA_t in Sections 3.2–3.6 and Figure 2), and reorganized Section 4 with explicit subheadings for protocol construction, metrics, mechanism analysis, and ablations.
Reviewer_SkmQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether an agent should use a stored memory snapshot when the fact's current validity is hidden and corrections arrive with a delay. RegimeCouncil scores each closed episode with and without memory, then routes between a Global Advisor (pooled over all keys), a Local Advisor (the queried key's last outcome) and a Meta Governor that defaults to Local and switches to Global on a positive recent surplus; three propositions motivate the design. On 16 LongMemEval knowledge-update questions with cached Qwen3-30B-A3B/GPT-4o answers and scheduled answer flips, RC beats max(G, L) by 5.2-7.3 points where the advisors disagree, with positive gaps on a 54-key pool and TempLAMA/TAQA switch times.

Critical strengths
  • A clearly scoped question with a clean testbed. The paper isolates how far a delayed correction should propagate: to one key or to the whole store. Stripping temporal cues makes validity truly hidden, and the observable-conflict case is kept out of the claims (App. B.10).

  • The router is simple, adds almost no cost and is fully interpretable. On Schedule A the gap decomposes exactly into helpful minus harmful overrides (10 vs. 2 on Qwen). Table 16 shows both failure modes of Prop. 2 on real items.

Critical weaknesses
  1. The delayed feedback is never actually tested. Answers are cached, the prompt tells the model the snapshot "records the correct answer" (App. C), and tau = 1 in every experiment, so staleness comes only from the 96-episode commit lag (App. B.2).

  2. Assumption 3 undercuts the motivation. Scoring both actions requires the gold answer, and with the gold the agent can simply write the correction back, as Mem0 and Zep already do and as Sec. 1 itself calls "straightforward".

  3. The gains are small, and the baselines are weak. On Schedule A the gap is 7-8 of 352 episodes, and all four Table 1 bootstrap CIs reach or cross zero (App. B.5). RC is compared mainly with its own components, not Fixed Share or partial pooling, and the simpler follow-recent-winner beats it on 19 of 30 settings (App. B.7).

Questions for authors
  1. How does the gap change when tau and the commit interval are varied separately (e.g. tau in {1, 16, 96})?

  2. When feedback only says the executed answer was wrong, how is u_0 obtained? When the gold is revealed, how does simply writing it back to the key compare with RC?

  3. Does RC still win against simple fallbacks on the same d_t stream, such as Local that defers to Global on ties or unseen keys, and Fixed Share? Please also report full-stream accuracy with CIs in Table 1.

  4. How were the 16 main keys chosen from the 54-key audited pool?

6 comments
AuthorsOpenReview ↗

Response to Reviewer SkmQ (Part1, Q1 & W1)

We sincerely thank Reviewer SkmQ for the thorough reading and constructive comments. We address all points in four comments:

Q1 & W1. Testing delayed feedback (τ>1\tau > 1), varying τ\tau and commit interval separately, cached completions, and prompt wording

In the submission, staleness came from the 96-episode snapshot commit interval while outcome revelation had τ=1\tau{=}1. To directly test delayed feedback, we vary outcome delay τ∈{1,4,8,16,48,96}\tau\in\lbrace 1,4,8,16,48,96 \rbrace and snapshot commit interval ∈{1,16,32,64,96,192}\in\lbrace 1,16,32,64,96,192 \rbrace independently across Schedules A, B, and C on both models, keeping the paper's fixed configuration (Wg=32,Wℓ=1,Wm=8W_g{=}32, W_\ell{=}1, W_m{=}8, gate 00).

  1. Commit interval at τ=1\tau{=}1 (mean full-stream accuracy, %). Since each of the 16 keys is visited once per 16-episode pass, commit ≤16\le 16 is immediate write-back before the key's next visit:
Commit intervalAlways memorymax(G, L)RegimeCouncil (submitted)RC vs Always / vs max(G,L)
1 or 16 (immediate write-back)93.092.691.6−1.4 / −1.0 (routing unnecessary; see Q2)
32 / 64 / 96 (paper) / 19287.7 / 78.9 / 70.1 / 51.286.4 / 77.5 / 69.6 / 56.786.9 / 79.4 / 71.9 / 56.7−0.8 / +0.5 / +1.8 / +5.5 vs Always (+0.5 / +1.9 / +2.3 / 0.0 vs max(G,L))
  1. Outcome delay τ\tau at fixed commit interval 6K6K (disagreement-slice gap vs max(G, L), points):
Pool (commit = 6K6K)τ=1\tau{=}1τ=4\tau{=}4τ=8\tau{=}8τ=16\tau{=}16τ=48\tau{=}48τ=96\tau{=}96
16-key (commit 96)+5.8+3.6−1.2−4.9−11.1−24.4
54-key (commit 324)+8.6+7.7+6.8+4.8+0.6−7.3
TempLAMA 120 / TAQA 100+11.5+11.0+10.1+8.8+2.60.0
  • Why the paper set τ=1\tau{=}1, and why performance is governed by τ/K\tau/K (Proposition 1(ii)): In the submission, each step tt represents a completed interaction and staleness was isolated in the 6K6K-step snapshot commit interval, so setting τ=1\tau{=}1 avoided confounding commit lag with feedback lag. When varying τ\tau explicitly, Proposition 1(ii) (RegretT/T≤(S+K)m/T\mathrm{Regret}_T/T \le (S+K)m/T, where m=⌈τ/K⌉m = \lceil \tau/K \rceil counts visits to the same key before its feedback arrives) shows that effective delay is governed by the ratio τ/K\tau/K (delay relative to the KK-step per-key revisit cycle):
    1. When feedback arrives within the key-revisit cycle (τ<K/2\tau < K/2) or varies randomly in-stream: Across τ≤4\tau{\le}4 at K=16K{=}16, τ≤48\tau{\le}48 at K=54K{=}54, and all τ≤96\tau{\le}96 at K=100–120K{=}100\text{--}120 (where users query ~100 attributes between two visits to the same fact), as well as under in-conversation random delays (Uniform{1..4}\mathrm{Uniform}\lbrace 1..4 \rbrace, {1..8}\lbrace 1..8 \rbrace, {1..16}\lbrace 1..16 \rbrace, and step 1→81{\to}8), the submitted RegimeCouncil maintains clear positive gains (+0.6 to +11.5 fixed; +5.9 to +7.2 median across 150 random-delay runs).
    2. Why K=16K{=}16 drops at extreme fixed τ∈{16,48,96}\tau\in\lbrace 16,48,96 \rbrace: With only K=16K{=}16 keys, τ=96\tau{=}96 equals the entire 6-pass snapshot refresh cycle—every key is queried 6 times before its first feedback arrives (m=6m{=}6), and during those unrevealed passes the submitted safety-first Local Advisor abstains (at=0a_t{=}0). If Local instead defers to Global while awaiting a key's outcome (your Q3 suggestion), the slice gap stays strictly positive (+5.9 to +12.3) even at τ=96\tau{=}96 across all four pools (126/126 settings).
  • Why cached completions test delayed feedback (τ\tau) faithfully: At step tt, the router's binary action at∈{0,1}a_t\in\lbrace 0,1 \rbrace only selects between the single-turn memory prompt and no-memory prompt, while τ\tau determines which past outcomes {ds:s≤t−τ}\lbrace d_s : s \le t-\tau \rbrace have arrived in Ft\mathcal{F}_t. Because single-turn LLM outputs depend only on the prompt for episode tt and not on internal router state, evaluating τ∈[1,96]\tau\in[1,96] on cached Qwen/GPT-4o completions produces the exact same routing trajectory as live calls while isolating routing from generation variance.
  • Prompt wording: All routers (Global, Local, RegimeCouncil) see the exact same prompt and cached completions, and standard RAG models likewise adopt a matching retrieved note under neutral headers (causing LongMemEval's >20-pt drop on knowledge-update items). We have clarified this in Appendix C and included the full delay and commit-interval evaluations in Appendix E of the revised PDF.
AuthorsOpenReview ↗

Response to Reviewer SkmQ (Part2, W2 & Q2)

W2 & Q2. Why Assumption 3 does not undercut the motivation (W2), obtaining u0u_0 when feedback only flags the executed answer (Q2a), and immediate write-back vs RegimeCouncil (Q2b)

  • W2 — Why Assumption 3 does NOT require a gold replacement fact or undercut the motivation: Assumption 3 only requires a binary evaluation signal dt=ut,mem−ut,0∈{−1,0,+1}d_t = u_{t,\text{mem}} - u_{t,0} \in \lbrace -1,0,+1 \rbrace (whether using memory helped or hurt), not a replacement gold fact to write back. In deployed assistants, feedback is often negative without a replacement value (e.g., "I don't live in Beijing anymore" or a rejected recommendation / failed tool call without stating the new city), or arrives while memory updates are held in an asynchronous batch consolidation queue (Mem0/Zep). In both cases, write-back cannot overwrite the key with a new gold value, yet the negative signal still identifies that the memory-based answer failed.
  • Q2a — How u0u_0 is obtained when feedback only says the executed answer was wrong:
    1. Asynchronous counterfactual check (no gold answer needed): When a negative signal or complaint arrives (e.g., "not in Beijing"), an agent pipeline can check a cached background no-memory response against that same constraint (whether the fallback avoided assuming Beijing) to obtain u0u_0 and recover paired dtd_t (71.6% full-stream accuracy, +7.3 median slice gain).
    2. Pure executed-only bandit feedback (no u0u_0 call at all): Even if only the shown answer is scored and dtd_t is estimated via inverse-propensity weighting (ε=0.05\varepsilon{=}0.05 exploration), RegimeCouncil still outperforms Global in 30/30 settings (+5.7 points) and edges out Local (+0.4 points), while paired scoring unlocks the full +4.5-point gain over Global.
  • Q2b — When the gold replacement IS revealed, how immediate write-back compares with RegimeCouncil (see commit-interval table above): If the user supplies the new gold value and the store synchronously overwrites the key before its next visit (commit <= 16 episodes), always using memory achieves 93.0% and routing is unnecessary. Governance is needed between revelation and consolidation (commit = 32, 64, 96, or 192 episodes) or when feedback lacks the replacement value, where always-memory drops from 87.7% to 51.2% and RegimeCouncil lifts accuracy by +0.5 to +5.5 points (+1.4 to +5.6 points with your Q3 tie-deferral). We have clarified this distinction in Section 1, Section 3.1, and Appendix E of the revised PDF.
AuthorsOpenReview ↗

Response to Reviewer SkmQ (Part3, Q3 & W3)

Q3 & W3. Same-dtd_t baselines (tie-deferring Local, Fixed Share, partial pooling) and full-stream CIs for Table 1

We ran all requested baselines on the exact same dtd_t stream across all 30 held-out settings (baseline hyper-parameters tuned on the 50 development streams; RegimeCouncil uses the paper's fixed Wg=32,Wℓ=1,Wm=8W_g{=}32, W_\ell{=}1, W_m{=}8, gate 00):

Method on the same dtd_t stream (30 held-out settings)Full-stream acc.Slice gap vs max(G,L): med / worstTotal harmful episodes (harm rate)
Follow-recent-winner (App. B.7)71.7+8.0 / +2.5746 (2.21%)
Fixed Share (dev-tuned η=2,α=0.3\eta{=}2,\alpha{=}0.3)71.7+8.0 / +2.5746 (2.21%)
CUSUM on surplus71.3+7.8 / +1.2746 (2.21%)
RegimeCouncil (submitted)71.6+7.3 / +0.6707 (2.09%, lowest harm)
Local deferring to Global on ties/unseen (your Q3 baseline, no Meta)72.4+9.7 / +4.9752 (2.23%)
Partial pooling (dev-tuned w=1,κ=0.25w{=}1,\kappa{=}0.25; reduces to same rule)72.4+9.7 / +4.9752 (2.23%)
RegimeCouncil + your Q3 tie-deferral (RC-refined: tie-deferring Local + Meta)72.3+9.8 / +4.9748 (2.21%)

Thank you for suggesting the tie-deferring Local rule—testing it revealed a clear accuracy–harm trade-off and a direct way to strengthen our paper:

  1. Why the submitted RegimeCouncil is effective (harm-minimizing operating point, outperforming tie-deferral under clustered shifts):
    • Lowest harmful exposure: Because abstaining (at=0a_t{=}0) guarantees zero harmful-memory exposure, the submitted RegimeCouncil's default of abstaining on ties (d=0⇒0d{=}0 \Rightarrow 0) achieves the lowest harmful memory exposure of all adaptive methods (707 harmful episodes vs 746 for Fixed Share/Follow/CUSUM and 752 for tie-deferring Local), while outperforming CUSUM (+0.3 pts) and matching Fixed Share within 0.1 pts.
    • Where submitted RegimeCouncil decisively outperforms standalone tie-deferring Local: When a cluster of corrections hits across keys (Figure 1, scene 2), each key's latest outcome is non-zero (d=+1d{=}+1) but stale. Standalone tie-deferring Local (without our Meta Governor) is locked onto that stale d=+1d{=}+1 signal: on 54-key synchronized-correction streams (sync_multi and sync_switch), standalone tie-deferring Local drops to 66.3% disagreement accuracy with 91 harmful episodes, whereas the submitted RegimeCouncil achieves 94.2% (+27.9 pts) and reduces harmful episodes from 91 to 37 (a 59% reduction).
  2. Incorporating your Q3 suggestion (RC-refined — trading a small increase in harmful episodes for a large accuracy gain):
    • Across the 30 held-out streams, ~81% of disagreements are ties/unseen keys (d=0d{=}0). Adopting your suggestion inside RegimeCouncil—letting Local defer to Global on d=0d{=}0 while keeping the Meta Governor (At>0A_t>0) for conflict arbitration—trades a modest increase of 41 harmful episodes across 33,792 decisions (from 707 to 748, i.e., 2.09% to 2.21% of episodes, still below standalone tie-deferral's 752 and matching Fixed Share's 746) for a substantial accuracy boost (+257 net correct answers, 72.3% full-stream accuracy, +9.8 median slice gain, outperforming Fixed Share, Follow, and CUSUM on 28–29 of 30 settings) while retaining 94.2% accuracy and reducing harmful episodes from 91 to 40 on clustered shifts. We have added both operating points (safety-first default vs. tie-deferring variant) to Appendix E.1 of the revised PDF and credited your review.

Table 1 with full-stream accuracies and key-cluster 95% CIs (5,000 replicates):

Schedule / ModelGlobalLocalRegimeCouncil (submitted) full / gap [95% CI]RC + Q3 tie-deferral full / gap [95% CI]
A, Qwen66.567.970.2 / +2.3 [0.0, 3.4]71.9 / +4.0 [1.4, 4.6]
A, GPT-4o67.371.373.3 / +2.0 [0.0, 3.4]75.3 / +4.0 [2.3, 5.4]
B, Qwen69.368.871.6 / +2.3 [−0.3, 4.0]72.2 / +2.8 [0.6, 4.3]
B, GPT-4o70.770.573.0 / +2.3 [−0.3, 3.7]73.3 / +2.6 [0.6, 4.0]
Jointly resampled over all four settings above68.569.672.0 / +1.9 [0.9, 3.1] (slice: +5.1 [2.3, 8.1])73.2 / +3.1 [2.0, 4.2] (slice: +8.2 [5.4, 11.5])

With only 16 keys, single-schedule key-bootstrap CIs have step size 1/16=6.251/16{=}6.25%; resampling the 16 keys jointly across the four Table 1 evaluations excludes zero cleanly for the submitted RegimeCouncil (+1.9 [0.9, 3.1] full-stream; +5.1 [2.3, 8.1] on the disagreement slice), and adding your Q3 tie-deferral makes all four individual-schedule CIs strictly positive as well. We have added these full-stream accuracies and CIs directly to Table 1 in the revised PDF.

Reviewer_ezkMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how memory-based agents should adapt when stored facts or user preferences change, focusing on whether feedback should influence only the relevant fact or memory-use decisions more broadly. It proposes a simple rule-based method with two advisors: Local decides whether to use memory based on the latest feedback for the queried fact, while Global uses recent feedback pooled across facts. A Meta Governor compares the scores each advisor would have achieved over a sliding window of completed interactions, following Global when its cumulative score is higher and Local otherwise. This comparison assumes that delayed feedback allows both the memory-based and fallback answers to be scored. The paper reports improvements over either advisor alone on controlled question-answering streams involving changing facts, including streams derived from personal-memory and temporal-knowledge benchmarks.

Critical strengths
  • The paper addresses an important problem for personal assistants: adapting memory use as user preferences and stored facts change over time.
  • The proposed strategy demonstrates empirical improvements over fixed local and global strategies across multiple evaluation settings, supporting the benefit of adaptively choosing between these two approaches.
Critical weaknesses
  • Limited insight beyond a heuristic design. Although the paper theoretically motivates combining local and global feedback, the proposed method remains a simple sliding-window selection rule. The analysis does not establish why this particular rule is preferable to other adaptive strategies, and the paper provides limited evidence connecting its design to observed failures in deployed personal assistants.
  • Limited analysis of adaptation to changing drift rates. The paper includes sensitivity experiments for the global window size, but does not systematically establish whether the fixed global and meta windows remain effective as the frequency of preference changes or feedback delays varies within a conversation. This limits confidence in the method’s robustness beyond the evaluated schedules. Strong feedback assumptions limit practical applicability. The Meta Governor requires correctness scores for both the memory-based and fallback answers, including the answer that was not selected. The experiments obtain these scores using benchmark ground truth and cached answers, with feedback available before the next decision. Real personal assistants may receive only sparse or ambiguous feedback on the answer actually shown. The paper does not demonstrate how the method would operate reliably under these more realistic feedback conditions.
  • Unmeasured computational overhead. The paper does not evaluate end-to-end latency, throughput, or inference cost. Although the routing logic is lightweight, its paired-feedback mechanism requires evaluating both the memory-based and fallback answers. The experiments rely on precomputed answers, leaving the cost of generating and scoring alternative responses in a live deployment unmeasured. Consequently, the practical tradeoff between accuracy gains and computational overhead remains unclear.
Questions for authors
  1. How robust is the fixed-window strategy when the rate of preference or fact changes varies over the course of a user’s conversation? Do the selected global and meta window sizes remain effective under such variation?

  2. Compared with using the Global Advisor alone, what additional inference and evaluation costs does RegimeCouncil incur, including the cost of obtaining paired feedback? Please report the accuracy–cost tradeoff in terms of latency, throughput, and model calls, clarifying which costs are shared by both methods.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ezkM (Part1, W1a & W1b)

Thank you very much for the constructive review and for the helpful suggestions. We address all points in four comments:

W1a. Why this rule is preferable to other adaptive strategies

Why generic rules fall short. Generic rules (Fixed Share, CUSUM, follow-recent-winner) treat the two advisors symmetrically and switch on short-term fluctuations. Here the two sources are structurally asymmetric:

  • Local: once key ktk_t is visited, Local is unbiased for ktk_t (zero cross-key interference).
  • Global: pools the K−1K-1 other keys, which may be in a different regime (Prop. 2).

Our rule. The router therefore anchors on Local by default (Wℓ=1W_\ell = 1, Prop. 1(ii)). Global overrides only when its surplus over the last Wm=8W_m = 8 revealed episodes is strictly positive, i.e., At>0A_t > 0. This filters out single-episode noise while still guaranteeing a switch within Wm+τW_m + \tau steps after a shift (Prop. 3).

Evaluation protocol. All methods run on the exact same dtd_t stream across three benchmarks:

  1. 50 development streams (5 families);
  2. 16 drift-rate shift settings (Q1 & W2a);
  3. 30 held-out settings.

RegimeCouncil uses the paper's fixed Wg=32W_g = 32, Wm=8W_m = 8 with zero retuning. To give the competing adaptive baselines the strongest possible footing, their hyperparameters are tuned on the 50 development streams.

Table 1. 50 development streams

StrategyFullSliceHarm ↓
Global78.082.91,422
Local78.684.4525
Fixed Share (η=2\eta = 2, α=0.3\alpha = 0.3)79.586.5893
Follow-recent-winner (App. B.7)80.288.3692
CUSUM on surplus79.887.2674
RegimeCouncil (submitted)80.488.7579

Table 2. Generalization: 16 rate-shift settings (Q1) and 30 held-out settings

StrategyRate-shift: FullRate-shift: Med. gapRate-shift: WinsHeld-out: FullHeld-out: Med. gapHeld-out: WinsHeld-out: Harm ↓
Global63.5−7.51/1667.1−2.07/302,273
Local66.50.015/1668.40.023/30680
Fixed Share67.6+3.115/1671.7+8.030/30746 (2.21%)
Follow-recent-winner67.6+3.115/1671.7+8.030/30746 (2.21%)
CUSUM on surplus67.7+3.315/1671.3+7.830/30746 (2.21%)
RegimeCouncil (submitted)68.1+4.316/1671.6+7.330/30707 (2.09%)

Takeaways.

  • RegimeCouncil is best on the 50 heterogeneous streams and on all 16 rate-shift settings.
  • On the 30 held-out settings it beats CUSUM on 17/30 and is within 0.1 pts of Fixed Share on full accuracy, without any retuning.
  • It has the lowest harmful exposure of all adaptive routers: 579 vs. 674–893 (development) and 707 vs. 746 (held-out).

W1b. Evidence connecting the design to failures in deployed personal assistants

Each module targets a documented failure mode in deployed assistants:

  1. Silent stale-attribute persistence (LongMemEval [Wu et al., 2025]): when a city or preference changes without an explicit delete, retrieval still surfaces the old record; prompt features (zt(z_t) cannot detect staleness (Prop. 1(i)), whereas our paired anchor (dt=umem−u0(d_t = u_{\text{mem}} - u_0) tests whether memory helped.
  2. Cross-attribute interference: Volatile attributes (project, travel city) coexist with stable ones (dietary rules, preferences). A global threshold suppresses valid stable memories after a few volatile-key errors—so we anchor on the key-isolated Local Advisor.
  3. Batch consolidation lag & life-event shifts (Mem0 [Chhikara et al., 2025]; Zep [Rasmussen et al., 2025]): batched consolidation leaves a cluster of keys stale after a relocation or job change; Global + Meta Governor (At>0A_t>0) pools cross-key evidence to halt harmful injection within (Wm=8(W_m{=}8) episodes.
AuthorsOpenReview ↗

Response to Reviewer ezkM (Part2, Q1 & W2a)

Q1 & W2a. Robustness to in-conversation variation in drift rates (col. 3 of Table 2)

  • How the 16 settings are built: Over 22 passes (commit window = 6 passes), per-visit flip hazard ptp_t shifts mid-stream (pass 11) in two directions—rate_up (from 0.040.04 to 0.300.30) and rate_down (from 0.300.30 to 0.040.04)—across 2 directions × 2 pools (16/54 keys) × 2 models × 2 seeds = 16 settings.
  • Why this range: (1) Protocol anchor: One flip per 6-pass commit window corresponds to an effective hazard of ≈0.14\approx 0.14/visit (and poisson_flip in Appendix B.6 Table 9 uses p=0.12p{=}0.12/visit); 0.040.04 and 0.300.30 bracket this baseline symmetrically. (2) Regime boundary (Prop. 1(ii)): p=0.04p{=}0.04 is near-static (≤1\le 1 flip over 22 passes; >78%>78\% valid per 6-pass window), whereas p=0.30p{=}0.30 is sub-commit high-churn (≈2\approx 2 flips per 6-pass window, so snapshots go stale before consolidation). (3) Assistant usage: It mirrors shifts between stable profile facts and fast-changing active tasks.
  • Wider ranges & shapes (no window retuning): Keeping the paper's exact window sizes (Wg=32,Wℓ=1,Wm=8)(W_g{=}32, W_\ell{=}1, W_m{=}8) and At>0A_t>0 rule unchanged (which are also positive across Wg∈{16,32,64}W_g{\in}\{16,32,64\} and Wm∈{4,8,16}W_m{\in}\{4,8,16\} in Appendix B.5, Tables 7–8), RegimeCouncil remains effective not only under the main shifts between 0.040.04 and 0.300.30 (68.1% full-stream accuracy, +4.3 points median disagreement-slice gain over the better single advisor, outperforming both Global and Local in all 16 of 16 settings), but also across wider and non-step drift schedules: bidirectional shifts between 0.020.02 and 0.200.20 (70.5% full-stream accuracy, +4.4 points median disagreement-slice gain over the better single advisor) and between 0.050.05 and 0.400.40 (63.7% full-stream accuracy, +2.5 points gain), three-stage shifts from 0.020.02 to 0.140.14 to 0.350.35 (66.2% full-stream accuracy, +2.4 points gain), and sinusoidal drift where ptp_t varies continuously between 0.020.02 and 0.350.35 (67.2% full-stream accuracy, +2.7 points gain).

AuthorsOpenReview ↗

Response to Reviewer ezkM (Part3, W2b)

W2b. How RegimeCouncil operates reliably under varying delays, sparsity, ambiguity/noise, and executed-only feedback

  • In-conversation delay variation (30 settings × 5 seeds = 150 runs, using the same fixed Wg=32W_g{=}32, Wℓ=1W_\ell{=}1, Wm=8W_m{=}8): under τt∼Uniform{1..4}\tau_t \sim \mathrm{Uniform}\lbrace 1..4\rbrace, {1..8}\lbrace 1..8\rbrace, {1..16}\lbrace 1..16\rbrace, and step τt:1→8\tau_t: 1 \to 8, RegimeCouncil keeps median slice gains over max⁡(G,L)\max(G,L) of +7.2 (zero negative cases in 150 runs), +6.7 (1 negative in 150), +5.9 (140/150 positive), and +7.2 (145/150 positive), versus 17 negative cases for Fixed Share / Follow at Uniform{1..16}\mathrm{Uniform}\lbrace 1..16\rbrace.

Three design properties ensure reliable operation (verified in Rows 1–4 below):

  1. Property 1 — Event-indexed windows under sparsity (Row 1): (Wg,Wℓ,Wm)(W_g, W_\ell, W_m) index the most recent revealed outcomes in Ft\mathcal{F}_t rather than wall-clock turns tt, so unrevealed turns never flush past evidence.

  2. Property 2 — Multi-step gating under ambiguity/noise (Row 2): Ambiguous turns map to dt=0d_t{=}0 (no directional push), while the Wg=32W_g{=}32 mean and the Wm=8W_m{=}8 gate (At>0A_t>0) filter isolated mislabeled outcomes.

  3. Property 3 — Two operating paths when feedback only targets the shown answer (Rows 3 & 4):

    • Path A (asynchronous counterfactual check; 0 ms added user latency; no gold labels — Row 3): When a correction signal arrives, the agent checks an asynchronously cached no-memory answer against the same correction signal. This recovers the paired utility difference dt=ut,mem−ut,0∈{−1,0,+1}d_t = u_{t,\mathrm{mem}} - u_{t,0} \in \lbrace-1, 0, +1\rbrace, which cancels per-query difficulty variance. Path A achieves 71.6% mean full-stream accuracy and a +7.3-point median disagreement-slice gain over max⁡(G,L)\max(G,L), with no held-out setting falling below max⁡(G,L)\max(G,L).

    • Path B (pure executed-action-only bandit; no background model call — Row 4): Only the shown answer is scored, and the paired difference is estimated by inverse-propensity weighting, d^t\hat d_t, with exploration rate ε=0.05\varepsilon{=}0.05 and the unbiasedness property E[d^t]=dt\mathbb{E}[\hat d_t] = d_t. This estimate alone prevents the Global Advisor from collapsing: RegimeCouncil attains 68.9% mean full-stream accuracy versus 63.2% for Global, achieves a higher score than Global in all 30 of 30 settings, and halves the number of harmful episodes per setting (31.3 versus 60.6). It also outperforms Follow-recent-winner in 23 of 30 settings (two-sided sign test p=0.002p{=}0.002) and Local in 18 of 30 settings. Comparing Row 4 with Row 3 shows that the paired anchor is worth +2.7 points of mean full-stream accuracy.

Empirical test (across all 30 held-out settings, no window retuning)MetricSubmitted RegimeCouncilBaseline comparison
Row 1 (Property 1 — Sparsity): outcomes revealed on only 75% or 50% of turns (5 seeds × 30 settings = 150 runs)Median disagreement-slice gain over max⁡(G,L)\max(G,L)+5.7 points at 75% (positive in 149 of 150 runs) and +3.2 points at 50% (positive in 130 of 150 runs)Outperforms Follow-recent-winner and Fixed Share (+5.5 points at 75% and +2.9 points at 50%)
Row 2 (Property 2 — Noise): 5% or 10% outcome label flip probabilityMedian disagreement-slice gain over max⁡(G,L)\max(G,L)+3.5 points at 5% noise and +1.0 point at 10% noiseOutperforms Follow-recent-winner and Fixed Share (+3.1 points at 5% noise and +0.7 points at 10% noise)
Row 3 (Property 3, Path A — Async check, 0 ms latency, no gold): shown answer and background no-memory answer checked on the same correction signalMean full-stream accuracy and median disagreement-slice gain over max⁡(G,L)\max(G,L)71.6% full-stream accuracy (+2.7 points over Row 4) and +7.3 points median slice gain (positive in all 30 of 30 settings; 23.6 harmful episodes per setting)Outperforms Global (67.1% accuracy) in all 30 of 30 settings and Local (68.4% accuracy) in all 30 of 30 settings
Row 4 (Property 3, Path B — Executed-only bandit): only the shown answer is scoredMean full-stream accuracy and harmful episodes per setting68.9% full-stream accuracy (31.3 harmful episodes per setting)Outperforms Global (63.2% accuracy, 60.6 harmful episodes per setting; winning in all 30 of 30 settings), Follow-recent-winner (68.5% accuracy; winning in 23 of 30 settings, p=0.002p{=}0.002), and Local (68.5% accuracy; winning in 18 of 30 settings)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.