- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors propose a method for maintaining long-term memory for embodied QA, with output for retrieval, answer or collect more data. They build a dataset (eval + train) and show improvements over the eval with their trained model, and also over other benchmarks.
Critical strengths
-
The problem of collecting enough data to answer embodied QA, maintaining long history and acting on more data collection is an important open problem.
-
The results over other benchmarks are important
Critical weaknesses
-
Important parts of the paper are not readable or unclear. For example:
- Sec 3 and sec 4 read like a is a list of parts and pieces without any clarity on how they fit together and what they are used for. It is impossible to assess the paper's contributions at this readability level.
- "preferentially evicted" - what does this mean?
- "A deterministic harness retains structured memory and historical images independently, enabling retrieval beyond the visual context" - how?
- "remaining budgets at round k" - what does this mean?
- "Update()" is a critical component of the proposed approach, yet it is not described at all in the paper, not the inputs, not the output, not the function itself.
-
I want to stress that I could not understand the paper even when reading it throughly, I could not understand the benchmark, how it compares to other benchmarks, the memory and action approach architecture etc.
Questions for authors
I ask for the authors to resubmit after a significant rewrite of the paper.