- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper presents SPARK, a multi-episode framework for correcting persistent embodied memory through physical re-observation. The framework consists of three main components: a semantic critic that identifies unreliable regions in an existing knowledge base, a targeted revisit planner that acquires additional observations, and a selective merge strategy that updates stored information by adding, removing, or replacing object entries. Unlike query-driven memory refinement, SPARK performs memory correction independently of downstream questions.
Critical strengths
-
The paper addresses an important problem in embodied memory systems: errors and incomplete observations in previously constructed knowledge bases can persist and affect downstream reasoning. Its focus on query-independent memory correction provides a meaningful perspective beyond conventional query-driven exploration and memory retrieval.
-
By acquiring new observations rather than relying exclusively on previously collected evidence, SPARK can recover information that was missed during initial exploration. The reported improvements in EQA demonstrate the practical potential of physical re-observation for improving persistent embodied memory.
Critical weaknesses
-
Although SPARK considers a relatively distinct setting of query-independent memory correction, closely related approaches have already investigated uncertainty-guided physical revisitation and reliable persistent memory updating. For example, SCOUT explores semantic uncertainty-driven revisitation for scene-graph refinement, while PBD-AG investigates evidence-based memory maintenance and active inspection. Compared with these approaches, SPARK primarily relies on heuristic diagnostic thresholds, fixed-offset viewpoint generation, and rule-based memory merging. The paper does not sufficiently establish how these mechanisms provide more effective error diagnosis, observation planning, or memory correction than existing alternatives.
-
Although the paper demonstrates EQA improvements from additional observations, the effectiveness of its core self-correction mechanisms remains insufficiently validated. First, Table 2 shows that selective merging achieves a slightly lower EQA gain than append-only updating (+0.080 vs. +0.082). While it reduces memory size, its advantage in accurately correcting erroneous entries remains unclear. Second, Table 3 reports only critic recall, without meaningful precision evaluation, making it difficult to assess whether the critic can reliably distinguish erroneous memory from correctly reconstructed regions.
-
The manuscript bears obvious traces of AI-generated writing and appears to contain copied content (e.g., the overuse of em dashes, a typical feature of AI-generated text). Please thoroughly review and revise the manuscript.
Questions for authors
-
What are the key technical advantages of SPARK over existing uncertainty-guided revisitation and evidence-based memory updating approaches? Can the authors provide direct or capability-matched comparisons to substantiate these advantages?
-
Given that selective merging does not outperform append-only updating in EQA, how can the authors demonstrate that it more reliably corrects erroneous memory? Can object-level correction accuracy, false deletion rates, and memory consistency be evaluated?
SCOUT and PBD-AG, merge vs. append, critic precision, writing
Thank you for pointing us to SCOUT and PBD-AG and for the thoughtful questions.
To reiterate the main point of the paper, SPARK adapts the corrective RAG principle to embodied RAG. Prior work such as Mind Palace, ReMEmbR, and Embodied-RAG does not treat the KB as something that can be wrong about what is in the scene. Instead of proposing one more method, SPARK is a whole correction loop, with diagnosis, revisits, merging, and its evaluation, built around an API-based perception system and an LLM/VLM-queryable KB, with the focus on the text that EQA retrieves.
We will cite both (SCOUT became public in June and PBD-AG in August, shortly before the deadline, while SPARK began long before), but they target different goals and are therefore not directly comparable, although they are complementary and welcome. For example, SCOUT can be categorized as active perception, building online a 3D scene graph of label probabilities and CLIP embeddings, and PBD-AG targets change tracking, following objects that move, appear, or disappear indoors. In contrast, SPARK corrects a text KB that already exists, so that later questions are answered from corrected content, and revisiting is only how it gets the evidence. More specifically, SCOUT relies on a still error-prone detector and its own label confidence, which needs open models like CLIP and produces no captions, while SPARK works with API-based models that produce text and diagnoses the stored KB itself. Main distinction being that neither stores an LLM/VLM-queryable KB from an earlier episode, nor measures whether EQA improves. SPARK's components can be improved, for instance with a modified SCOUT traversal strategy, but a better first pass still cannot rule out errors, and SPARK is about the errors that do get stored. However, we already name better viewpoint selection and navigation as limitations and online correction as future work (Sec. 7).
At first, these look interchangeable, since the selective merge already performs deduplication, please refer to Algorithm 3 in the Appendix. Getting the same EQA as append is not a weak result, because NLP memory research shows that keeping the full history is the strongest baseline (Mem0, Chhikara et al., 2025, Li et al., 2024). However, a properly queryable RAG needs a healthy KB, and one would not want to keep appending corrupted information to it. From our data append and deduplication keep all 86 objects the critic flagged as implausible across both conditions. The selective merge, on the other hand, removes 46, which is every single one a revisit had in view but could not visually confirm, such as three hallucinated "security cameras", a "sandbag", and two "electrical junction boxes". EQA does not show this, because questions are created automatically only from the oracle view, which could not produce these hallucinations. This removal also depends on the critic, since the merge only removes objects the critic flagged, so without the critic all 86 would stay in the KB, as with append.
Instead of "undefined", it is rather that precision is 100% and hardly informative, because no region is adequately recovered, even in the undegraded reference memory (where the critic flags 63% of regions), so false positives cannot occur (Appendix C). Regarding the thresholds, like the merge rules, they are simple on purpose and fixed across maps, so that gains and failures can be traced.
As our AI use statement says, we used AI tools to help with language editing, while the writing and its manual editing are ours, and the em dashes were our own stylistic choice. The manuscript does not contain copied content, and we would be grateful if the reviewer could point us to any specific passage of concern so that we can check it.
Between submission and now we kept improving the work, and we uploaded a revised PDF. The main change is that the captioning prompt earlier relied on a coarse vocabulary of 22 object types, a common way to match the evaluation vocabulary, and now labels are fully free text, grouped per map. Other changes are (1) the selective merge no longer discards object types seen once in a region, which had removed correct objects, and (2) object placement now uses AirSim's perspective depth instead of the planar formula. The main EQA gain rises from +0.112 to +0.216, as expected after these fixes, and not every change favoured us, since the depth fix lowered the low-lateral gain.