- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents the Janus framework to represent belief and history for Multimodal Large Language Model (MLLM) guided exploration. The Janus framework consists of two parts, the Hierarchical Spatial Belief module and the Belief-Conditioned Evidence Imagination module. The Hierarchical Spatial Belief module builds a spatial graph for information retrieval for the query. The Belief-Conditioned Evidence Imagination module predicts the next feature for next-view selection. Experiments show that such modules could enhance MLLMs performance on embodied geo navigation benchmarks. The ablation study verifies the effect of each specific module.
Critical strengths
I cannot identify the strength since many details are missing.
Critical weaknesses
- [Poor Writing] The symbols in the method section lack definitions and explanations. I will list some examples below:
- In section 4.2, what is the shape of the visual feature ? How do you encode it?
- For line 231, what is the confidence score? How do we get it? And how do we get the supporting-view references?
- For line 251-252, how do you revise the claim? More specifically, how do you update the confidence score and the references?
- Equation 6, what is and ? If they are networks, do you train them or just use pre-trained checkpoints?
- Equation 7, how do you compute U?
These definitions and explanations are critical for the understanding and evaluation of this paper.
- [Lack of Baseline] The result table only shows LLM vs. LLM + Janus without other baselines that use memory for LLM on these tasks. The author discusses several previous works that use memory but does not compare to them in the experiments. It is unclear whether the improvement is brought by the proposed belief and imagination mechanism or simply historical information.
Questions for authors
See critical weakness.