- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses two failure modes in multi-agent LLM shared memory: misleading credit assignment when a reader correctly answers using parametric knowledge instead of retrieved incorrect facts, and stale reputation that persists after a writer changes behavior. The authors propose Priced Reputation Learning (PRL), which discounts positive credit using the reader's closed-book success probability and applies exponential forgetting to adapt to behavioral shifts. Supported by theoretical analysis on reputation lock-in, extensive empirical evaluations and supplementary experiments demonstrate that PRL consistently improves source selection and approaches oracle performance.
Critical strengths
-
The paper conceptually distinguishes raw task success from actual memory usefulness, highlighting how parametric knowledge can cause naive systems to reinforce incorrect sources. Coupling this with the temporal issue of stale reputation provides a highly coherent, theoretically grounded motivation for PRL's specific mechanisms, rather than relying on arbitrary heuristics.
-
The supplementary material bolsters the main paper by comprehensively testing complex threat models like redundant/conflicting memories, multi-step reasoning, colluding writers, and reputation farming. The comparison with counterfactual replay baselines proves that PRL not only improves credit assignment but does so at a fraction of the computational cost, confirming the method is both highly effective and efficient.
Critical weaknesses
-
The credit signal is a task-level heuristic rather than principled causal attribution. While the closed-book baseline effectively corrects the specific leakage phenomenon identified, it merely discounts expected reader-alone success rather than causally isolating which specific retrieved fact produced the outcome. Although supplementary experiments show this approximation works well empirically even in multi-fact settings, the conceptual limitation remains: PRL is essentially a task-level reputation correction rule, not a fundamental solution to fine-grained, causal memory attribution.
-
Limited conceptual novelty and broader scope. The core components of PRL—baseline discounting and temporal forgetting—are individually standard concepts in credit assignment and reputation systems. While their specific combination effectively solves the constructed LLM memory benchmarks, the paper lacks a sharp conceptual characterization of what fundamentally new principle is being introduced beyond simply applying these existing ideas to a new domain. Consequently, the theoretical formulation feels somewhat narrow in scope despite its empirical success.
Questions for authors
-
Is PRL intended primarily as a computationally efficient approximation to counterfactual memory attribution, or does the reader-conditioned pricing rule represent a fundamentally different theoretical formulation? Clarifying this would help assess the method's novelty.
-
Given that the task-level baseline cannot determine which specific served fact produced a success, do you view this as a fundamental limitation of PRL, or do you expect this pricing principle could be extended to fact-level attribution?
-
Given that the closed-book probe and the memory-enabled task use different prompts and output formats, is the baseline intended to measure the probability of solving the underlying task generically, or the probability of producing a strictly matched answer?
-
Where exactly does PRL sit within the emerging design space of shared multi-agent memory? How does PRL conceptually position itself against the rapidly emerging literature on multi-agent reliability memory, particularly frameworks like Σ-Mem?
-
Is the reader-conditioned pricing mechanism fundamentally specific to "shared memory," or is it a more general framework for outcome-based LLM credit assignment?
3 comments
Q2. Fundamental limitation, or extensible to fact level?
The pricing principle extends directly to individual writers in multi-writer reads, and we implemented and tested it. Even at the task level, PRL already ranks every reliable writer above every unreliable one across all three multi-fact FEVER benchmarks. When finer attribution is needed on a multi-writer read, we replace the empty-memory probe in Equation 1 with a probe leaving one writer out: writer receives on success, where probes the reader with the other served writers' facts kept in context. In a controlled co-serving test where colluding writers are co-served on every read and share credit on every success, this per-writer price selects the correct version on 97–100% of attacked reads and matches the writer oracle within paired confidence intervals on every seed (Section 3.2, Table 5, and Appendix H.6).
Q3. What does the baseline measure?
We design our baseline to measure the probability that the same reader produces an answer accepted by the exact same task checker without memory. Because the closed-book probe is graded by the identical checker used for the task with previous memory (), the baseline and the task outcome lie on the exact same strict-match scale. While the task with memory runs the full multi-agent workflow and the probe queries the reader directly, the two agree closely when memory is empty (63.0% for the single-call probe vs. 60.8% for the full workflow with empty memory; Appendix H.7). This gap is negligible relative to the safety margin in Section 3.4: avoiding lock-in requires a mean baseline above 0.31, and the measured value on PopMix is 0.84–0.86. Increasing the number of probes per task does not improve full PRL either, yielding mean accuracies of 91.8%, 91.4%, and 90.8% for across three seeds (Appendix H.4).
Q4. Position relative to Σ-Mem and the design space.
PRL modifies the credit signal from where reliability is learned, which is orthogonal to how reliability state is parameterized. In the broader design space, agreement-based methods fail under false majorities, raw outcome-based updates suffer from reader-knowledge lock-in, and counterfactual replay requires hundreds of extra model calls. Because pricing operates on the feedback signal itself, it serves as a plug-in improvement across different reliability representations on the same store, including our adaptation of Σ-Mem (Table A2):
| PopMix accuracy (%), Qwen3 8B | Raw outcome credit | Priced credit | Gain |
|---|---|---|---|
| Beta reputation per writer | 66.3 | 86.7 | +20.4 |
| Credibility Scoring (multiplicative score per writer) | 65.4 | 90.8 | +25.4 |
| Σ-Mem adaptation (decayed reliability per writer and relation) | 80.4 | 85.4 | +5.0 |
Full details and a structured design-space comparison are provided in Appendix H.7.
Q5. Specific to shared memory?
PRL is a general credit-assignment principle. The same confounding arises whenever a consumer agent is sufficiently competent to succeed on its own and outcome feedback determines which source is selected next, such as retrieval source ranking, tool or peer routing, and multi-agent debate aggregation. Shared memory is where the failure is most severe, because stored sources persist across tasks and selection feedback compounds the leak into permanent lock-in.
Thank you for the detailed rebuttal. It has substantially addressed my concerns. I maintain my original score of 3.
Thank you for the careful reading and constructive feedback.
Our contribution is identifying a new problem, explaining its theory, and providing its solution. Writer reputation learned from task outcomes fails in two ways. First, a reader that already knows the answer succeeds whatever it is served, so its success rewards the served source even when that source is false. Second, the evidence a writer earned while it was reliable outlives a change in its behavior. We identify both failures, derive the exact conditions under which each occurs, and remove them: PRL credits a source only for what the reader could not achieve without it, and lets old evidence expire.
For W1: The price is a well-defined causal contrast, and it estimates the exact quantity that writer selection requires. Specifically, the credit 1−p^0(xt) compares two outcomes of the same reader on the same task: its performance with the served memory versus without it. That contrast measures the net effect of serving memory on task xt. Whether selection favors or filters out a false source depends on the sign of that source's expected reputation drift, and Section 3.4 derives the exact condition under which pricing makes this drift negative. Moreover, whenever a read serves a single fact (97.5% of scored PopMix reads), this task-level price coincides with the leave-one-out effect of that specific fact.
For W2. "Limited conceptual novelty."
Our manuscript introduces a new failure formulation, a theoretical analysis of when it occurs, and a coupled update rule derived directly from that analysis.
1. A previously unexamined failure mode. In shared memory, outcome-based credit is confounded by the reader's own competence: a reader that already knows the answer succeeds regardless of what it retrieves, and that success rewards the served writer even when its fact is false. Existing reliability rules fail under this leak: unpriced Beta reputation, iP-TD, and both Credibility Scoring variants serve the correct fact on only 0.3% of attacked PopMix reads.
2. Exact lock-in conditions. Section 3.4 derives closed-form conditions for when a false source locks in under unpriced credit (q>1/2), under priced credit (q>1/(2−pˉ0)), and under any constant price. Verified against recorded traces, these thresholds accurately predict both where unpriced rules lock in PopMix experiment and where no lock-in occurs because the reader cannot solve the task alone in TOFU and MQuAKE experiments.
3. Why the update rule differs from a variance baseline. Unlike a variance reducing baseline in policy gradients, reputation here is a ratio of accumulated positive and negative evidence. Pricing therefore flips the sign of the expected drift and shifts the fixed point of source selection. Pricing and forgetting also play non-interchangeable roles: the lock-in threshold is independent of λ, while pricing alone cannot clear evidence accumulated during an earlier honest phase. In our new four-phase experiment with three reliability reversals (Appendix H.5), only the full PRL rule recovers the newly reliable writer after every reversal, whereas pricing alone and forgetting alone never switch back.
Q1. Approximation of counterfactual attribution, or a different formulation?
It is a fundamentally different formulation rather than an approximation. Post-hoc replay attribution aims to explain a single past outcome fact by fact, whereas PRL is an online learning rule designed to govern future source selection. Accordingly, our theoretical guarantee characterizes the drift of writer reputation under repeated selection (Section 3.4), a property that per-task attribution does not address. Empirically, on the same memory store, PRL is both more accurate and substantially cheaper than counterfactual replay: