- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes LoRAM (Learned Role-Aware Agentic Memory), which uses role-structured persistent memory to provide clear responsibility division. Each agent maintains its local episodes and consolidated procedures. And a global transactive directory is used to store provenance, compatibility information, and other metadata. The authors use a compatibility gate to filter peer sources and retrieve from separate candidate sets. A contextual LinUCB policy selects among bypass, local, single-peer, and local-plus-peer access under the same memory budget. For evaluation, they use 100-task streams from three MultiAgentBench environments (Research, Database, and Coding), where LoRAM achieves the highest mean scores and reduces token usage compared with other memory frameworks.
Critical strengths
- The concrete design and formulation are logically motivated. In particular, LoRAM proposes and clearly separates several useful components: role compatibility, task expertise, transfer history, and search relevance. It also combines compatibility gating with UCB routing over different actions, which is reasonable since the router needs to balance using sources that have worked well before with exploring sources whose utility is still uncertain. These constitute a meaningful architectural contribution to persistent shared memory.
Critical weaknesses
- The experimental results are not sufficiently convincing. The authors evaluate LoRAM on three different environments, while the main improvements are relatively small, especially on the Research and Coding tasks. These marginal improvements are also not shown to be stable through confidence intervals or variance estimates. In addition, since this is a sequential and evolving process, task order may also matter. Evaluating alternative task orders could therefore help assess the stability of the reported improvements.
- Some experimental designs make it difficult to attribute the gains to learned source routing. For example, full LoRAM and LoRAM-Local do not have access to equally sized retrieval spaces. LoRAM-Local searches only the target agent’s own memory, whereas full LoRAM additionally retrieves top-K candidates from each eligible peer, resulting in a larger candidate set. The adapted item-routing baseline is also not a fully controlled comparison. It uses a fixed heuristic scoring function, while LoRAM learns a scorer and additionally chooses among bypass/local/cross-agent/hybrid access. The LoRAM-StaticRole comparison could help isolate the value of online learning, but its exact fixed routing rule is not specified.
Questions for authors
-
Could the authors specify the "fixed routing policy" used by LoRAM-StaticRole? Specifically, how does it select the access mode and peer source?
-
Could the authors provide a more controlled ablation on the number of retrieved candidates across LoRAM-Local and full LoRAM? For example, the two variants could have access to the same amount of memory, while one pool contains only local memories and the other includes memories from eligible peers. This would help separate the benefit of more diverse cross-agent memory from simply having access to a larger retrieval corpus.
-
Could the authors report results under different task orders or provide uncertainty estimates such as confidence intervals? Given the small improvements across the three environments, a performance stability check would be very important.
3 comments
Thank you for the detailed rebuttal.
The response has addressed several of my concerns. But I still have some reserverations: (1) Although the gains are consistent across the three task orders, they remain very small. (2) The robustness analysis is still limited to the Research environment.
I prefer to maintain my rating.
Thank you for your follow-up and for recognizing that our rebuttal has addressed several of your concerns. We appreciate your careful consideration of the additional experimental results.
Regarding the remaining concerns, we would like to highlight that LoRAM demonstrates improvements across all three environments, with larger gains in Database and Coding. As reported in Table 2, LoRAM improves Database accuracy from 35.00 to 40.00 (+5.00 percentage points) and Coding performance from 69.01 to 71.51 (+2.50 points) over LLMA-Mem. These results complement the Research findings and provide evidence of LoRAM's effectiveness across different task settings.
During the rebuttal, we specifically prioritized Research for additional robustness evaluations and controlled ablations, as its smaller performance margin made it a particularly important setting for examining the robustness and contributions of adaptive source routing. The new experiments show consistent positive improvements across three matched task orders, together with additional evidence from component-level comparisons.
Given the limited rebuttal period, we concentrated our experimental efforts on these targeted Research evaluations rather than repeating the full multi-order study across all three environments. We believe the combination of the original cross-environment results and the newly added Research experiments provides meaningful and complementary evidence supporting LoRAM's effectiveness.
Thank you again for your constructive feedback and consideration.
Response to Reviewer S6bc
We sincerely thank the reviewer for recognizing the motivation and architectural contributions of our work. We address the concerns regarding experimental robustness, routing-policy design, and retrieval fairness through additional experiments and clarifications below.
W1 / Q3 — Task-Order Robustness and Limits of Stability Evidence
We evaluated Full LoRAM and the baselines on three matched permutations of the same 100 Research tasks with GPT-4o-mini. Memory, directory, and routing state were reset between orders, allowing each method to learn independently on the same task sequence.
Table B1. Matched Research order checks (GPT-4o-mini; 100 tasks/order).
order_00(original)order_01order_02As shown in Table B1, Full LoRAM consistently outperforms both StaticRole and LLMA-Mem across all three matched task orders, achieving average improvements of +0.347 J and +0.267 J, respectively. The positive margins across different task permutations demonstrate that LoRAM's performance improvements are reproducible across the evaluated task sequences, providing additional empirical support for the effectiveness of adaptive source routing.
W2 / Q1 — Exact StaticRole Routing Rule and What It Controls For
The fixed routing rule used in our comparison is described as follows.
StaticRoleretains the same profile-initialized compatibility gate (θ_role=0.55), source-preserving local/peer top-Kretrieval (K=5), transfer eligibility restriction to consolidated peer procedures, and common memory-token budget (B=1,000). It does not select actions using learned LinUCB weights. Instead, it applies a deterministic access rule:Hybrid(i,j*), wherej*is the eligible peer with the largest fixed prior role-profile similarity (ties broken by fixed role index).Cross(j*)by the same role-priority rule.Local.Bypass.Memory cards are still packed under the same
B=1,000budget. This comparator tests whether a task-conditioned online router is useful beyond a simple deterministic role-priority heuristic. It should not be interpreted as a gate ablation or as the same control asFrozen:Frozenkeeps the LinUCB action-scoring equation active while holding its learned parameters fixed, whereasStaticRolereplaces that scoring process with a deterministic action rule.W2 / Q2 — Candidate-Pool Size and Fairness of Full Versus Local-Only Retrieval
To examine retrieval space size directly, we audited 1,000 logged decisions across
order_01andorder_02. Each decision contained on average 4.20 local candidates and 8.80 eligible peer-procedure candidates; 24% of local decisions had fewer than five local records. This motivates a comparison with a common maximum number of pre-budget candidates.For the
order_00control, each variant receives a single global cap ofC≤5candidates across its combined sources, followed by the same memory-token budgetB=1,000. The cap is global, rather than five candidates per source.Table B2. Global candidate-cap comparison (Research; GPT-4o-mini;
order_00).C≤5)Under the shared cap, Full achieves +0.39 J over Local-only and +0.16 J over LearnedItem. This experiment controls the maximum candidate count and the final prompt-memory budget. Actual nonempty candidate counts still reflect store contents—for example, early local memories can contain fewer than five records—so we interpret this as a global candidate-cap-controlled comparison, with retrieval diversity remaining part of the system design.