- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Meta-MemGran, a framework that uses structured memory to improve the visual representations of DINOv3 for multimodal reasoning. The method builds four memory banks for global information, entities, spatial layouts, and relationships, and combines their features through prototype retrieval and memory aggregation. Experiments on several benchmarks show improvements in reasoning performance and hallucination reduction. The overall motivation is reasonable, and the results are promising. However, I still have some concerns about the novelty of the method, several implementation details, and whether the experiments sufficiently demonstrate the claimed adaptive multi-granularity capabilities.
Critical strengths
- The paper addresses a relevant challenge in MLLMs: bridging fine-grained pixel-level perception and high-level semantic reasoning.
- The proposed Meta-MemBank incorporates four components covering global context, entities, spatial layouts, and inter-entity relationships. This provides a structured approach to organizing visual information, and the combination of prototype retrieval and memory composition offers a potentially useful alternative to explicit granularity control through an external language model.
- The paper reports experiments across multiple benchmarks and language model backbones, including Qwen and Llama models of different sizes. The results show improvements over several visual encoder baselines, particularly on reasoning and GPT-based evaluation metrics.
Critical weaknesses
The authors highlight that Meta-MemGran can adaptively organize visual information without relying on an external LLM for granularity control. However, the method mainly builds on existing techniques, including online clustering, prototype retrieval, spatial relation modeling, and attention-based feature fusion. Similar ideas have been explored in prior work on visual token aggregation and memory-augmented learning. As a result, I find the novelty somewhat limited, as the main contribution seems to be the combination of four types of visual memory rather than a new mechanism for adaptive granularity learning.
The Entity Bank uses entity names extracted from paired captions to organize visual prototypes. However, these captions are not available during inference, where the model only receives an image and a question. I am therefore unclear about how the Entity Bank works at inference time. How are the relevant entity names identified, and how are the entity indices used for retrieval?
I am not sure whether the four memory banks capture sufficiently different information. In Eq. (4), the Global Bank averages all patch features, while the Entity Bank averages the cluster centers. These representations may be quite similar, making it unclear how the Entity Bank preserves individual object information. Similarly, the Relation Bank relies on linear combinations of cluster features and predefined spatial rules, which may capture basic spatial relationships but struggle with more complex interactions. Although the ablation results show that the four banks contribute to performance, further analysis would help clarify whether they actually capture distinct and complementary semantic information.
The paper claims that Meta-MemGran can adaptively organize visual information based on different task requirements, but I would like to see more evidence of this. For example, when the same image is paired with questions about the overall scene, individual objects, or spatial relationships, do the retrieved memory prototypes and the contributions of different memory banks change accordingly? It would be helpful to show how the retrieval results and aggregation weights vary across these questions to better support the claimed adaptive multi-granularity capability.
Questions for authors
Please refer to the Critical Weaknesses section.