- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies long-term memory for multimodal agents, with a particular focus on tracking how persistent entities evolve across multiple sessions while retaining access to the original visual evidence. The authors argue that existing abstraction-based memory systems may discard fine-grained visual details, whereas pixel-preserving systems often fail to associate visually diverse observations with the same entity or to organize its state changes over time.
To address this problem, the paper proposes MHG-Mem, an entity-centric multimodal hypergraph memory. It represents entity mentions, atomic facts, and aspect-specific state evolution as different nodes, and connects them using typed occurrence, evolution, and evidence hyperedges. A hypergraph-based agentic search procedure selects an appropriate entry layer, traverses relations across memory levels, and inspects the original dialogue or image when fine-grained verification is needed. The method is evaluated on MemEye, Mem-Gallery, and WorldMemArena-S with three backbone models. The reported results consistently outperform the included memory baselines, and the ablations attribute the gains to entity grouping, explicit evolution modeling, provenance links, and adaptive search.
Critical strengths
-
The paper identifies a meaningful gap at the intersection of multimodal memory and temporal state tracking. Prior work has studied multimodal memory and evolving or temporal memory separately, but it remains difficult to associate visually diverse observations with a persistent entity, track its changes across sessions, and retain the original evidence needed for fine-grained verification. This is a practically relevant failure mode for long-running multimodal agents.
-
The proposed representation is coherent and is supported by reasonably comprehensive component ablations. The occurrence, evolution, and evidence hyperedges have clear functional roles, and the agentic retrieval procedure can move between compact state summaries and original observations. The consistent improvements across three benchmarks and three backbone models suggest that this combination is useful within the evaluated setting.
Critical weaknesses
The novelty is limited and is primarily an A+B-style integration. Entity-centric memory, multimodal evidence retention, temporal state tracking, provenance links, hypergraph-based organization, and agentic retrieval have each been explored in prior work. The main novelty lies in combining these ideas for multimodal entity evolution. This is a sensible design choice, but the paper does not establish a sufficiently large conceptual or algorithmic advance beyond the combination itself. In particular, a stronger comparison against the closest graph-based or provenance-aware multimodal memory systems, as well as against simple extensions that add temporal tracking to an existing multimodal memory baseline, would be necessary to demonstrate that the proposed hypergraph formulation is essential.
- The evidence for generality, scalability, and robust performance is limited. The method is closely tailored to benchmarks in which entity evolution and evidence tracing are important, and the evaluation does not establish how well the full pipeline transfers to substantially different domains, longer and noisier histories, or open-ended agent environments. The system also depends on several model-generated stages, including entity association, fact extraction, evolution synthesis, and agentic retrieval. Errors in entity linking or state summarization may propagate through the graph, but these intermediate components are not independently evaluated. The additional construction, storage, and retrieval costs are also not compared with the baselines on an accuracy-cost frontier. Consequently, it remains unclear whether the observed gains justify the added complexity or generalize beyond the current evaluation setting.
Questions for authors
-
Can the authors provide a controlled comparison that isolates the proposed joint representation from a simple combination of existing techniques? For example, how does MHG-Mem compare with (i) a strong multimodal memory baseline augmented with an explicit temporal state table, and (ii) a temporal or graph memory baseline augmented with access to the original images and provenance pointers, under the same backbone, information access, and model-call budget? This comparison would materially affect my assessment of whether the typed hypergraph provides benefits beyond an A+B integration.
-
How robust is MHG-Mem to errors in entity association and evolution synthesis? Please report the accuracy of entity linking, state-update extraction, and evidence-edge construction, together with an error analysis showing how these errors affect downstream QA. A controlled perturbation experiment that introduces incorrect identity links or conflicting state updates would also clarify whether the method remains reliable in realistic noisy histories.
-
Can the authors provide stronger evidence of scalability and generalization? In particular, please report performance and cost as the number of sessions, entities, and observations grows, including memory construction calls, storage size, retrieval latency, and token usage. Results on an additional domain or a less evolution-centric benchmark would help determine whether the method is a generally useful memory architecture or mainly a specialized solution for the evaluated task structure.
2 comments
The current comparison does not disentangle the benefit of the proposed hypergraph memory from that of additional inference-time computation. Full MHG-Mem consumes 31,407 tokens, compared with 16,762 tokens for UniversalRAG, while agentic search alone contributes 14,696 additional tokens and 5.10 Judge points. A compute-matched UniversalRAG baseline equipped with iterative agentic retrieval is therefore necessary to determine whether the remaining gain arises from the proposed memory representation and traversal mechanism, rather than from a larger inference budget.
Looking forward to your feedback
Dear Reviewer LHWd,
As the rebuttal deadline is approaching, we are eager to know whether our responses adequately address your concerns. We would be happy to clarify any additional questions you may have.
We have also incorporated additional experiments, analyses, and clarifications into the revised PDF, including:
We would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our work. It has been a pleasure engaging with you throughout this process.
Best regards,
The Authors