acceptodds
Under review as a conference paper at ICLR 2027

SuperMem: Omni-Native Super-Hypergraph Memory for LLM Agents

Abstract

Long-term memory lets a frozen large language model (LLM) agent answer from experience beyond its context window, increasingly omni-modal streams of text, images, audio and video. Agent memory has grown from flat stores to hypergraphs and from text to all modalities, yet hypergraph memories index only text while all-modal ones stay flat or pairwise. This leaves three challenges: modality collapse in storage, the modality gap in recall, and repetition and revision in update. We propose SuperMem, an omni-native super-hypergraph memory whose hyperedges bind each unit's text with its native media nodes and nest units into episodes, with no extra LLM call. Joint cross-modal recall credits each modality above its own per-query median and delivers whole units with raw media, while conservative consolidation merges repeats and supersedes revisions without deleting. On nine text, multimodal and omni-modal benchmarks, SuperMem has the best average against 13 memory baselines on each of three frozen backbones, 9.5 points above the strongest.

Primary area:
applications to neuroscience & cognitive science
Keywords:
agent memory, multimodal memory, hypergraph

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3xeVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SuperMem, an omni-native super-hypergraph memory framework. The framework consists of three main components: (1) omni-native super-hypergraph memory construction, (2) joint cross-modal recall with sufficiency gating, and (3) conservative consolidation without deletion. These components aim to address three challenges in long-term multimodal memory: modality collapse in storage, modality gaps in retrieval, and repetition and revision during memory updates. The proposed method demonstrates strong performance across nine text, multimodal, and omni-modal benchmarks using three frozen backbones, outperforming existing memory baselines in average performance.

Critical strengths
  • Strong performance and comprehensive evaluation across diverse benchmarks and backbones. The proposed method demonstrates strong performance across nine text, multimodal, and omni-modal benchmarks using three different frozen backbone models. SuperMem consistently achieves the best average performance compared to existing memory-based methods. The extensive comparisons with 13 baselines, along with ablation, efficiency, and robustness analyses, further support the effectiveness of the proposed framework.

  • Unified omni-native memory with no additional LLM calls. The proposed framework integrates hypergraph-based memory with omni-native representations, while constructing and consolidating memory through deterministic operations without additional LLM calls beyond shared media rendering. This provides a practical approach to organizing and maintaining long-term omni-modal memory.

Critical weaknesses

Major Weaknesses

  • Insufficient justification for the super-hypergraph structure. Although the paper provides an arity-based ablation for first-level hyperedges (Appendix F.3), it remains unclear what advantages the proposed super-hypergraph offers over conventional hierarchical graph-based memory. The unit-level and episode-level relationships could potentially be represented using a hierarchical graph structure. Furthermore, the contribution of the second-level hyperedges is not independently evaluated. Appendix D.2 states that the store is emptied for each episode before writing its stream. Since each stream corresponds to one episode and the default setting uses $k_2=8$, episode-level filtering appears to have no effect under this evaluation protocol. This raises questions about the necessity and effectiveness of the proposed two-level super-hypergraph structure.

Minor Weaknesses

  • Inconsistent citation. In Lines 143–144, the citation (Lee et al., 2025) refers to the earlier Gemini Embedding model, whereas Lines 335–336 specify Gemini Embedding 2 [1] as the encoder used in the experiments. The citation should be updated accordingly.

  • Figure clarity and consistency. The two bar charts in Figure 2(d) appear to present identical results, making the visualization redundant. Additionally, the boundaries between first-level and second-level hyperedges in Figure 3(a) are somewhat ambiguous compared to Figure 2(c). A more consistent visualization would help clarify the proposed memory structure.

  • Clarity of writing. Although the overall framework is understandable, some sentences are difficult to follow due to complex phrasing and dense mathematical notation, particularly in the methodology section. Simplifying these explanations would improve the readability of the paper.

[1] Shanbhogue, Madhuri, et al. "Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini." arXiv preprint arXiv:2605.27295 (2026).

Questions for authors
  • Evaluation subset selection. The paper evaluates 128 questions from each of the nine benchmarks (Table 5), but the selection criteria are not clearly described. How were these questions selected, and how representative are they of the original benchmarks? Clarifying the sampling strategy and reporting the category-wise question distribution would help assess the representativeness of the evaluation results.

  • Super-hypergraph structure. Could the authors clarify what advantages the proposed super-hypergraph provides over a conventional hierarchical graph that connects episode, unit, and media nodes? Additionally, given the per-episode evaluation protocol in Appendix D.2, how does the second-level hypergraph contribute to retrieval performance? A comparison with a conventional hierarchical graph under the same native media access and retrieval settings, or an evaluation involving multiple streams in the same memory store, would help clarify its necessity.

  • Sufficiency gate. How was the sufficiency threshold θd=0.45\theta_d=0.45 determined? How frequently does the gate filter retrieved evidence across the benchmarks, and how does its removal affect performance? Since sufficiency-gated delivery is one of the main components, these results would help clarify its actual contribution.

  • Conservative consolidation. The repeated-write experiments demonstrate the effectiveness of consolidation in preventing memory duplication and preserving retrieval performance. However, it remains unclear whether MERGE correctly identifies semantically equivalent information and whether SUPERSEDE accurately handles revised or conflicting information. An analysis of the frequency and correctness of these operations, along with concrete examples of memory updates, would help better understand the effectiveness of the proposed consolidation mechanism.

  • Hyperparameter selection. The proposed framework involves several heuristic hyperparameters across memory construction, retrieval, and consolidation. How were these hyperparameters selected? In particular, were they tuned on a separate validation set or using the same 128 evaluation questions from each benchmark? Clarifying the hyperparameter selection protocol and whether the evaluation questions were kept separate from tuning would help assess the robustness and fairness of the evaluation.

  • Qualitative examples and failure analysis. While the paper provides a successful retrieval case study in Table 4, additional examples of the constructed memory, including first-level and second-level hyperedges, would help clarify how the proposed structure works in practice. Furthermore, an analysis of failure cases would help better understand the limitations of the proposed framework.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 3xeV (Part 1/2)

Thank you very much for your time and thorough review, and for recognizing the strong performance, comprehensive evaluation and unified omni-native memory with no additional LLM calls. Your concerns are the super-hypergraph vs. a hierarchical graph, the second level if "the store is emptied for each episode" (App. D.2), the sufficiency threshold and the gate, how the questions were selected, how representative they are, the correctness of MERGE and SUPERSEDE, hyperparameter tuning, examples of the constructed memory, failure cases, and minor weaknesses.

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in blue (mainly p.9 and App. F.8–F.9), and red marks changes addressing points raised by several reviewers. Part 2 answers Q4 to Q6 and the minor weaknesses.


W1: Super-hypergraph vs. a hierarchical graph

Thank you very much. We added the hierarchical graph you suggested (HierGraph) on the same store (new RQ8, §4.9, Fig. 10a; Table 17, p.29): the super-hypergraph wins, most clearly in one store holding all nine benchmarks.

  • HierGraph keeps SuperMem's episodes, units, nodes, encoder, k1k_1/k2k_2, gate and delivery, and scores a unit by its best node (max) or the mean. R@5 counts gold units in the top five, Own@1 whether the top unit is from the asked episode (media sets), and accuracy covers the six MM and Omni sets:
VariantR@5Own@1Acc.R@5 (all-nine store)Own@1 (all-nine store)
HierGraph-mean (no end-to-end run)80.673.2–33.327.2
HierGraph-max84.574.236.764.556.4
SuperMem86.678.440.575.468.2
  • Joint scoring helps: each modality above its per-query median adds a bonus to a unit's text match. Likewise, the pairwise restriction reaches 78.0 R@5 against 86.6 (Table 12, Fig. 5).

Therefore, HierGraph-max trails the super-hypergraph by 2.1 R@5, 4.2 Own@1 and 3.8 accuracy points, by 10.9 R@5 in the 14,329-unit store (363 episodes), and by 5.6 to 18.8 on every media set there.


Q2: The second level if "the store is emptied for each episode"

Thank you very much. We revised the Write-then-query sentences that open App. D.2 (p.22, in blue): App. D.2 now reads "A benchmark's episodes share one store".

  • Every experiment writes all episodes of a benchmark into one store before answering (8 to 128 episodes per store, Table 5), so episode-level selection always faces many competitors.
  • The episode bounds consolidation (Eq. 9) and the span of the temporal prior (Eq. 6), and the frozen k2k_2 lies on a plateau within 0.3 R@5 of its neighbours (Table 15, Fig. 9a).

Hence, the second level scopes updates and time, and recall is robust to it.


Q3: The sufficiency threshold θd=0.45\theta_d=0.45 and the gate

Thank you very much. θd\theta_d equals the frozen file floor θfile=0.45\theta_{\mathrm{file}}=0.45 (Table 9, p.25), so one threshold serves delivery and file hand-back. We tested the gate signal (new in §4.10, App. F.9, Table 19) with the median s⋆s^\star per benchmark:

QuestionsnMedianAUROC
In store1,1520.70 to 0.96–
Asked episode masked1,1520.55 to 0.700.71 to 0.99
Another benchmark's store (new here)7680.61 to 0.700.79 to 0.997
  • Wrong-episode evidence misleads the backbone: shuffled recall (Placebo) falls below No memory by up to 21.8 points (MemEye, Table 6, p.22).

As a result, the score the gate thresholds separates in-store from out-of-store questions on every benchmark (0.98 or more on LoCoMo, MemEye and LongDocURL).


Q1(a): How the questions were selected

Thank you very much. The subsets are drawn as follows:

  • Seven benchmarks use one fixed seed, stratified by their own question type and spread over episodes.
  • M3-Bench-robot and LVOmniBench first pick videos covering every scenario or question type (17 of 100 and 19 of 275).
  • MemoryAgentBench takes 32 questions per competency by design, and LongDocURL and OmniVideoBench are uniform random draws with a fixed seed.

Therefore, every subset follows a fixed, stated rule.


Q1(b): How representative they are

Thank you very much. Each subset's category mix is much closer to the full release than a same-size random draw (total-variation distance, lower is closer):

SubsetLoCoMoLongMemEvalMem-GalleryMemEye
Random 128 (median / p95)0.059 / 0.1060.063 / 0.1070.090 / 0.1350.085 / 0.123
Our 1280.0040.0030.0090.011

Within their chosen videos, M3-Bench-robot and LVOmniBench match almost exactly (0.004 and 0.007). Hence, results reflect each benchmark's own question mix.


AuthorsOpenReview ↗

Response to Reviewer 3xeV (Part 2/2)


Q4: Correctness of MERGE and SUPERSEDE

Thank you very much. We added controlled write probes on fresh stores (new in App. F.8, pp.28–29; in teal, shared with Reviewer HAeo): all 88 exact repeats merge into their own edge and all 88 unrelated facts are inserted.

  • MERGE: rewriting each stream of LoCoMo, Mem-Gallery and MemEye up to 8×8\times merges every repeat, so store size and R@5 equal those of a single write, while an append-only store grows 8×8\times and loses 26.4 R@5 (§4.6, Fig. 7, Table 13).
  • SUPERSEDE: when a near-identical unit changes a number+unit slot (Eqs. 9–10), the new unit is inserted and the old one leaves the live set; it is still recalled for "before / originally" questions (Eq. 11) and never deleted (Prop. 3). In LoCoMo conv-41 (verbatim), "He was such an important part of our family for 10 years", rewritten with 11 years, supersedes the original unit.

As a result, no merge or supersede hits a wrong unit.


Q5: Whether hyperparameters were "tuned on a separate validation set or using the same 128 evaluation questions"

Thank you very much. We set the knobs during development on these benchmarks and then froze them everywhere (Table 9; App. E.6 notes one backbone-specific exception). We also added a re-run and one-knob changes (new, §4.10, Fig. 11b; App. F.9):

  1. Three re-runs of the released code give identical answers and average 39.2 (Table 2: 39.9, from earlier runs), against 30.1 for the strongest baseline.
  2. Changing one knob at a time gives 39.2 with k1=5k_1=5, 37.4 with wmed=0.5w_{\mathrm{med}}=0.5 and 36.6 with wlex=0w_{\mathrm{lex}}=0, and a 2k- or 8k-character note budget gives 31.9 or 36.7 (Fig. 8a), all above 30.1.
  3. Recall is flat once k1≥6k_1 \ge 6 and across k2k_2, peaks at wlex=0.2w_{\mathrm{lex}}=0.2 and rises with wmedw_{\mathrm{med}} (Table 15, Fig. 9a).
  4. In paired tests, SuperMem beats each baseline on 64.2% to 83.5% of questions where they disagree (sign test, p<10−8p<10^{-8}; Table 16, Fig. 9b).

Therefore, every changed value stays above the strongest baseline.


Q6(a): Examples of the constructed memory

Thank you very much. The case study is now RQ10 (§4.11): Table 4 (p.10), App. F.10 (p.29) and Fig. 14 (p.18) trace one recall from the M3-Bench-robot store (276 first-level hyperedges in 17 episodes, Table 12).

  • Episode bedroom_07 is one second-level hyperedge nesting 16 first-level hyperedges, one per two-minute clip.
  • Each binds 3 text nodes (subtitles with the shared rendering, plus passages), 3 frames 40 s apart and the clip's audio (arity 7; 6 for the last, shorter clip).
  • Fig. 14 prints the eight delivered notes with their per-modality similarities.

Hence, the example runs from episode to delivered notes.


Q6(b): Failure cases

Thank you very much. Of SuperMem's 693 wrong answers on gemini-2.5-flash-lite (Table 2), most lie in the backbone's reading or in telling similar episodes apart:

  • 64.4% came after the right evidence was delivered.
  • 30.3% have the top unit from a confusable episode. For example, "Which book does Lucy like?" (M3-Bench-robot) ranks a clip of study_03 above the asked study_06.
  • 5.3% miss the gold evidence in the right episode.

Therefore, recall rarely misses inside the right episode.


Minor weaknesses

Thank you very much. We revised all three:

  • Inconsistent citation: p.3 (§3, after Eq. 1) now cites Gemini Embedding 2 (Shanbhogue et al., 2026) instead of Lee et al. (2025), matching the encoder of every run, gemini-embedding-2 (§4.1, p.6; Table 8, p.23).
  • Figure clarity and consistency: Fig. 2(d) is now one horizontal chart naming each backbone's strongest baseline (p.2). Fig. 3(a), like Fig. 2(c), now labels the dashed episode (second-level) hyperedge and the shaded first-level hyperedges (p.4).
  • Clarity of writing: each module of §3 now opens with one plain sentence naming its challenge, Table 1 (p.3) contrasts one representative per design, and Algorithm 1 (App. C, p.21) walks through the three phases.

Hence, all three are now fixed.


Summary of Revisions

WhereChange
App. D.2, opening Write-then-query sentences (p.22)One store per benchmark, shared by all its episodes (revised, in blue)
§4.9 (RQ8), Fig. 10a; App. F.8, Table 17New: HierGraph in per-benchmark and all-benchmark stores
§4.10 (RQ9), Fig. 11b; App. F.9, Table 19New: re-run, one-knob changes, gate separation
§4.11, App. F.10Case study, now RQ10
Fig. 2(d), Fig. 3(a)One horizontal chart; labelled first- and second-level hyperedges

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score further. Thank you very much!

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns, and most of my questions have been addressed. However, it remains unclear what specific advantages the super-hypergraph structure provides over a conventional hierarchical graph, as the new comparison also differs in its retrieval scoring mechanism. I will therefore maintain my original score.

Reviewer_HAeoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SuperMem introduces an omni-native super-hypergraph memory for LLM agents to handle multimodal streams (text, images, audio, video) without extra LLM write costs. It addresses storage modality collapse and the recall modality gap by binding native media nodes into first-level hyperedges and nesting units into second-level episodes. Using a median-anchored joint cross-modal recall and conservative consolidation (merging repeats/revisions without deletion), SuperMem outperforms 13 memory baselines across nine benchmarks.

Critical strengths

SuperMem offers a more holistic solution, compared to previous methods, to managing long-term memories for AI agents that operate in an omni-modal environment (processing video, audio, images, and text simultaneously) and require high-precision and cost-efficient retrieval of raw perceptual evidence across long-horizon timelines.

The effectiveness of SuperMem is convincingly validated via evaluations.

Critical weaknesses

Zero LLM write cost, due to its relying on deterministic layout and embedding logic rather than expensive LLM-driven graph extraction, can be both a good feature, and a limitation e.g., it cannot perform any distillation like Mem0 or PlugMem, thus may lead to storing more, repetitive nodes than necessary.

Questions for authors

As mentioned above, it is useful to discuss the weakness of the method, esp. its reliance on deterministic math, rule-based hyperedge creation, and pre-computed embeddings, and storing incoming chunks, passages, and media raw or via predefined templates.

In addition to showing the superiority of SuperMem over other methods, it is also useful to discuss pros and cons of different paradigms for managing long-term mem in LLMs and AI agents. For example, while many of these methods mentioned in the paper, such as SuperMem, GraphRAG, PlugMem, address the limits of RAG, they approach data structure, modality, and agent integration quite differently, and are optimized for different settings.

1 comment
AuthorsOpenReview ↗

Response to Reviewer HAeo

Thank you very much for your time and thorough review, and for recognizing SuperMem as a more holistic, convincingly validated solution. Your concerns are repetitive nodes without distillation, the trade-offs of the rule-based design, and the pros and cons of memory paradigms.

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in teal, and red marks changes addressing points raised by several reviewers.


W1: Without distillation, "more, repetitive nodes than necessary"

Thank you very much. We added a write-cost and redundancy analysis (new RQ8, §4.9, Fig. 10b, p.9; App. F.8, pp.28–29): SuperMem makes 0 LLM write calls and near-duplicates are rare. Over 14,329 written units (gemini-2.5-flash-lite logs, shared rendering r(m)r(m) excluded):

MemoryLLM write calls (per unit; USD)
Zep159,833 (11.2; 6.15)
Mem0$^g$154,708 (10.8; 7.64)
Mem092,206 (6.4; 5.22)
A-Mem28,651 (2.0; 2.95)
HyperMem14,330 (1.0; 2.70)
G-Memory11,681 (0.8; 13.00)
SuperMem0
  • 1.5% of edges (213 of 14,329) have a near-duplicate in their episode at cosine ≥0.97\ge 0.97 (7.5% at ≥0.90\ge 0.90), filling 1.1% of top-5 units delivered for 1,152 questions.
  • Shared files and repeated facts are stored once (§3.1; Prop. 3, Eq. 17). Writing each stream 8×8\times leaves store and R@5 unchanged, while an append-only store grows 8×8\times and loses 26.4 R@5 (§4.6, Fig. 7, Table 13).
  • Delivery is capped (App. C). At a 2k-character budget SuperMem averages 31.9, above every Table 2 baseline, even V-Mem's 30.1 at 5,480 characters (§4.7, Fig. 8a). Its first 500 characters hold 58.5% of gold evidence, more than any baseline's first 32,000 (Fig. 8b, Table 14).
  • Distillation may delete evidence (§1; Rusu et al., 2026). In the case study, HyperMem's text-only fact yields "Information not available." and SuperMem's clip "Right" (§4.11, Table 4).

Therefore, SuperMem keeps few repetitive nodes without distillation and averages 39.9 on gemini-2.5-flash-lite, against 22.7, 18.3 and 15.4 for the distilling Mem0, Mem0$^g$ and Zep (Table 2).


Q1: Trade-offs of the rule-based design

Thank you very much. App. H (Future Work, p.30) now discusses deeper nesting, new modalities with their own median (keeping Prop. 2), an LLM summary node and shared stores. Each rule-based choice has a trade-off:

  1. Image scores trail text by 0.18 to 0.28 (§4.5, Table 11). The median-anchored bonus (Eq. 5) is immune to a constant gap (Prop. 2), and joint recall beats text-only at all 21 sensitivity settings, by 2.3 to 7.9 R@5 (Table 15).
  2. Hyperedges are sessions, pages or clips as written, without LLM segmentation. Binding native nodes gives 86.6 R@5, against 78.0 for pairwise links (Table 12) and 84.5 for a hierarchical graph (§4.9).
  3. Updates follow a fixed rule (Eq. 10), call no LLM and delete nothing (Prop. 3). In probes (App. F.8), 88/88 exact repeats merged, 88/88 unrelated facts were inserted, and no update hit a wrong unit.
  4. Raw storage makes deletion explicit via source files (Ethics Statement, p.11).
  5. Embeddings are cached (§3.1), so a new encoder needs one re-embedding pass and no LLM.

Hence, each choice keeps memory LLM-free and deterministic.


Q2: Pros and cons of memory paradigms

Thank you very much. Each paradigm suits a different setting, and SuperMem targets long-horizon multimodal evidence; App. G (p.30) covers GraphRAG and PlugMem qualitatively.

  • Distilled (Mem0, MemGPT, PlugMem): compact and suit assistants recalling user facts, but call an LLM at write and lose raw evidence and timing.
  • Graph and text-hypergraph (GraphRAG, Zep, A-Mem, G-Memory; HyperGraphRAG, HyperMem, MAGE): add multi-hop or high-order relations but see media only as text.
  • Omni-native: need no LLM at write (Table 7) but link media differently. V-Mem routes a query to one lane, M3-Agent draws one binary edge per file (45.4 mean R@5 vs. 86.6), and SuperMem one hyperedge per unit.

As a result, V-Mem, MemOS and M3-Agent lead Text, MM and Omni among baselines, while SuperMem leads all three (§4.2, Table 10) with one LLM-free store for any backbone.


Summary of Revisions

WhereChange
New RQ8 (§4.9), Fig. 10b (p.9)Write cost; rare near-duplicates
App. F.8 (pp.28–29)Write calls; near-duplicates; write probes
App. H (p.30)Future Work, incl. an LLM summary node

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score further. Thank you very much!

Reviewer_dvWtOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work proposes SuperMem, an omni-native agent memory whose hyperedges bind each unit's text with its native media nodes and nest units into episodes. On various omni-modal benchmarks, SuperMem has the best average against 13 memory baselines on each of three frozen backbones, 9.5 points above the strongest.

Critical strengths
  1. Hypergraph structure is proper for the setting where multi-modal datas are grouped.
  2. The agent memory show stronger performance without extra LLM calls.
Critical weaknesses

None

Questions for authors
  1. Will the memory system also works for frontier models like opus 5.5 and astra?
1 comment
AuthorsOpenReview ↗

Response to Reviewer dvWt

Thank you very much for your time and thorough review, and for recognizing the hypergraph structure as proper for the setting where multimodal data are grouped, and its stronger performance without extra LLM calls. Your concern is whether the memory system also works for frontier models like Opus 5.5 and astra.

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in purple, and red marks changes that address points raised by several reviewers. You will find them in the new RQ9 on frontier backbones (§4.10, Fig. 11a, p.9; App. F.9 and Table 18, p.29), in the red RQ list (§4, p.6), the caption of Fig. 11 (p.9) and the last sentence of the Conclusion (p.10).


Q1: "Will the memory system also works for frontier models like opus 5.5 and astra?"

Thank you very much. Yes. The memory does not depend on the backbone, and SuperMem lifts both frontier models on every benchmark we ran (new RQ9, §4.10, Fig. 11a; App. F.9).

  • New runs: Claude Opus 5.5 and GPT-6-astra replace the backbone on one benchmark per block, with the released code, stores, recall, delivery and prompts frozen; memory off delivers nothing. Both read text and images, and audio and video reach them through the shared rendering. We build the store once and reuse it unchanged, so no backbone-specific write is needed. Each is one run on the stated subset, beside the three full backbones of Table 2.
  • Results on stratified subsets of the 128 test questions (seed 20261108; token F1 on LoCoMo, accuracy elsewhere), in new Table R9 (= Table 18, p.29):
BackboneBenchmark (n)Memory offSuperMemGain
Claude Opus 5.5LoCoMo (40)28.457.1+28.7
Claude Opus 5.5MemEye (40)50.087.5+37.5
Claude Opus 5.5M3-Bench-robot (40)0.017.5+17.5
GPT-6-astraLoCoMo (32)8.640.6+32.0
GPT-6-astraMemEye (16)31.281.2+50.0

Four memory-off replies of Opus 5.5 were refused by the provider and count as wrong. Four observations:

  1. Frontier models still need memory, because without it they cannot know what happened in earlier sessions or recordings. Opus 5.5 scores 0.0 on M3-Bench-robot and declines to answer 26 of its 40 questions, and with SuperMem the gain is 17.5 to 50.0 points.
  2. A stronger reader makes better use of the same evidence. On the same MemEye questions, SuperMem reaches 87.5 with Opus 5.5 and 81.2 with GPT-6-astra, against 55.0 and 56.2 with gemini-2.5-flash-lite, while the store and recall stay unchanged.
  3. A long context is no substitute. On a 1M-context backbone, SuperMem beats replaying the whole stream in the prompt on five of seven columns (Table 6, p.22), while its own context stays bounded (App. C).
  4. Readers with fewer native modalities still gain. Both frontier models read only text and images (App. F.9), yet on M3-Bench-robot, whose evidence is video and audio, Opus 5.5 gains 17.5 points through the shared rendering.
  • Why it transfers: writing and recall call no LLM (§3.1; App. F), so the store and recall are the same for every backbone. Only the frozen reader changes, and media it cannot read reach it through the shared rendering (Table 8). Every row also shares the answering prompts (App. A.2, Fig. 13).
  • Backbones already in the paper: across three with different input modalities (Table 8), SuperMem has the best average on each: 33.3, 39.2 and 39.9 against 28.2, 26.3 and 30.1 for the strongest baseline (§4.2, Table 2). These are leads of 5.1, 12.9 and 9.8 points, and +17.8, +24.7 and +21.9 over No memory. Averaged over backbones, it scores 37.5 against 28.0 for V-Mem, the strongest baseline (the 9.5-point margin of the abstract), and it is the best row on all nine benchmarks (§4.2, Fig. 4).

Thank you for this question; RQ9 now answers it in the paper. Therefore, the memory system also works for the frontier models you named.


Summary of Revisions

WhereChange
§4, RQ list (p.6, red)New RQ9: "Does it generalize to frontier backbones?"
§4.10 (RQ9), Fig. 11a (p.9)New: Claude Opus 5.5 and GPT-6-astra, memory off vs. SuperMem
App. F.9, Table 18 (p.29)New: setup and per-benchmark results
Fig. 11 caption (p.9, red)Panel (a): frontier backbones, memory off vs. SuperMem
§5 (p.10, red)"It also beats a hierarchical graph and lifts frontier backbones."

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score further. Thank you very much!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers,

Thank you very much for your time and thorough reviews. We are glad that you found "strong performance and comprehensive evaluation across diverse benchmarks and backbones" and a "unified omni-native memory with no additional LLM calls" (3xeV), saw SuperMem as a "more holistic solution" whose effectiveness is "convincingly validated" (HAeo), and agreed that the "hypergraph structure is proper for the setting where multi-modal datas are grouped" with "stronger performance without extra LLM calls" (dvWt). Your main questions concern the structure, the cost of raw storage and frontier backbones, and we answer each below.

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite all reviewers to open the revised PDF to view them. The revised PDF has 30 pages: the main text on pp.1–10, statements and references on pp.11–16, and Appendices A–H on pp.17–30.

Each reviewer's revisions have their own colour, blue for Reviewer 3xeV, teal for Reviewer HAeo and purple for Reviewer dvWt, and red marks changes that address points raised by several reviewers (the overlapping concerns). Blue covers the HierGraph paragraph of §4.9, the robustness and gate sentences of §4.10, the Write-then-query sentences that open App. D.2, App. F.8–F.9 and Tables 17 and 19. Teal covers the write-cost sentences of §4.9, the write-cost, near-duplicate and probe sentences of App. F.8, and App. H. Purple covers the frontier-backbone paragraph of §4.10, App. F.9 and Table 18. Red covers the RQ list of §4, the captions of Figs. 10 and 11, the RQ10 headings of §4.11 and App. F.10, and the last sentence of §5.

Summary of our revisions:

Concern / changeWhere in the revised paperReviewer(s)
New research questions RQ8 and RQ9; the case study is now RQ10RQ8: §4.9, Fig. 10, App. F.8; RQ9: §4.10, Fig. 11, App. F.9 (p.9 and pp.28–29); case study: §4.11all
Hierarchical graph vs. super-hypergraph: 86.6 vs. 84.5 R@5 and 40.5 vs. 36.7 accuracy; 75.4 vs. 64.5 R@5 in one store holding all nine benchmarks§4.9, Fig. 10a; Table 17 (blue)3xeV
Role of the second level: every experiment uses one store per benchmark, shared by all its episodes (8 to 128 per store, Table 5), as App. E.1 (p.24) also states; recall is robust to k2k_2 (plateau within 0.3 R@5 of its neighbours); the second level scopes consolidation and the temporal prior of each episodeOpening Write-then-query sentences of App. D.2 (p.22, in blue); Table 15, Fig. 9a3xeV
Sufficiency gate: the gated score separates questions whose episode is absent from the store (AUROC 0.71 to 0.99; 0.98 or more on LoCoMo, MemEye and LongDocURL)§4.10; Table 19 (blue)3xeV
MERGE correctness: exact repeats 88/88 merged, unrelated facts 88/88 insertedApp. F.8 (teal)3xeV, HAeo
Cost of not distilling: 0 write calls vs. 92,206 (Mem0) and 159,833 (Zep); near-duplicates: 1.5% of edges, 1.1% of delivered top-5 units§4.9, Fig. 10b; App. F.8 (teal)HAeo
Robustness: re-run 39.2 (Table 2: 39.9, earlier runs); 36.6 to 39.2 with one knob changed, vs. 30.1 for the strongest baseline§4.10, Fig. 11b; App. F.9 (blue)3xeV
Frontier backbones: on one benchmark per block, SuperMem adds 17.5 to 50.0 points to Claude Opus 5.5 and GPT-6-astra§4.10, Fig. 11a; Table 18 (purple)dvWt
Constructed memory: one recall traced through bedroom_07, with 16 first-level hyperedges nested in one second-level hyperedge§4.11, Table 4; App. F.10, Fig. 143xeV
Distillation as an extension: an LLM summary of a unit can join its hyperedge as one more text nodeApp. H (teal)HAeo
Sampling; failures: TV 0.003 to 0.011 vs. 0.059 to 0.090 for a random 128; 64.4% of errors occur after the right evidence was deliveredthis discussion3xeV
Presentation: redrawn figures; one colour palette; ten research questions; Fig. 2(d) is now one horizontal chart naming each backbone's strongest baseline; Fig. 3(a) draws the episode (second-level) hyperedge as a dashed outline and each first-level hyperedge as a solid shaded region, both labelledFigs. 1–3 (redrawn), Figs. 1–11 (palette); §4 (p.6)all
Protocol: every new experiment uses the released code (anonymous link in the paper) under the frozen configuration; analyses of existing runs use the runs of Table 2, and all baseline numbers are those of the paperTable 9all
Reproducibility: writing and recall call no LLM, so an offline replay with the released code reproduces a run's recall exactly; every run summary carries a hash of the frozen configuration; three re-runs of it give identical answersApp. F; Table 9; App. F.9all

Each reviewer's thread below gives the full evidence and ends with its own table of changes. If any point is ambiguous or deserves further discussion, we would be happy to elaborate. Thank you again for your valuable time!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.