- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper considers a lifelong object navigation setting where the agent repeatedly searches for small household objects in the same environment, while some objects may be moved between episodes without the agent observing the change. The authors name this setting LiSoNav-COL and introduce LiSoNav-Eval, a benchmark with 28 scenes, 1,199 subtasks, and 45 object categories. Around 45% of the subtasks involve relocated targets.
The proposed method, IVAM-Nav, builds a persistent graph memory around historical viewpoints. During exploration, the agent actively observes supporting surfaces from several viewpoints and stores object/surface information together with the corresponding camera poses. For later tasks, it retrieves relevant historical viewpoints, revisits them, checks whether the stored information is still valid, and updates stale local memory when necessary.
The experiments compare IVAM-Nav with several navigation and scene-memory baselines. IVAM-Nav reaches 33.4% overall SR, compared with 27.5% for the strongest reported oracle baseline, and achieves 30.5% SR on relocated targets. The ablations further show that complementary viewpoints, graph aggregation, historical-viewpoint revisit, and memory updates all contribute to the final performance.
Critical strengths
- The problem setting is well motivated and not fully covered by existing benchmarks.
The paper combines several issues that matter in practice: small objects are hard to observe, the agent starts without a prebuilt scene memory, past experience should help future tasks, and that memory can become stale when objects are moved. I think this is a meaningful setting for lifelong embodied navigation.
- The method is simple enough to understand, and the main design choices are backed by experiments.
The strongest part of IVAM-Nav, in my view, is the use of historical viewpoints as memory anchors. Instead of only storing that an object was seen near a certain place, the method also keeps track of where the observation came from and later returns to a similar viewpoint for revalidation. The gains are also fairly substantial. IVAM-Nav outperforms the reported baselines in overall SR and performs especially well on relocated targets. The ablations and the separate revalidation experiment make the improvement easier to attribute to the proposed components rather than only to the overall system complexity.
Critical weaknesses
- The type of environmental change considered in the benchmark is still fairly limited.
The benchmark changes the locations of small objects, while the supporting surfaces and the rest of the background scene remain stable. The authors also mention this limitation themselves.
This matters because the viewpoint-anchored memory assumes that returning to a historical viewpoint is still meaningful. In a real home, furniture can move, supporting surfaces can change, localization can drift, and lighting or clutter can change significantly. It is therefore not yet clear how well the same mechanism would work under broader scene changes.
I do not see this as a fatal issue for the current paper, but it makes the real-world motivation somewhat stronger than the current evaluation can fully support.
- The paper does not give enough information about the computational cost and long-term scaling of the method.
IVAM-Nav uses active multi-view inspection, repeated historical revisits, and MLLM calls at several stages. SPL captures path efficiency, but it does not tell us how expensive the system is in terms of inference time, number of MLLM calls, or memory growth.
This is particularly relevant because the paper presents the problem as lifelong navigation, while the benchmark sequences only contain three or four episodes. The authors show that the number of retrieved historical candidates grows across episodes, and they also acknowledge that multi-view observation introduces additional navigation cost.
I would have liked to see at least a basic runtime and memory analysis, or an experiment showing how the method behaves over much longer sequences.
Questions for authors
- Is there a fully held-out test split?
The appendix describes the 1,199-subtask set as the full LiSoNav-Eval validation set, and the method-analysis subset is sampled from it.
Could the authors clarify whether the reported 28 scenes were also used during method development or hyperparameter selection? Is there a separate held-out test split that was not used during development?
This would affect my confidence in how much of the reported gain generalizes beyond this particular benchmark instance.
- How expensive is IVAM-Nav in practice?
Could the authors report the average number of MLLM calls, wall-clock runtime, and memory size per subtask or per episode? It would also be useful to show how these quantities grow as the number of episodes increases. Since the method repeatedly revisits historical viewpoints and actively collects multiple views, this seems important for evaluating the practicality of the “lifelong” setting.
- How sensitive is the method to the particular MLLM used?
The benchmark construction uses Qwen-VL-Max for annotation/verification, while IVAM-Nav uses Qwen3-VL for several semantic and decision-making steps.
Have the authors tested another MLLM, or at least a different set of independently written target/context descriptions? I would be interested in knowing whether the gain is stable across model choices and language styles.
- The related work could be expanded a little.
There is prior work studying how navigation agents acquire and exploit environmental knowledge from different perspectives, including learning navigation and layout priors from large-scale real-world videos [1], object-centric search based on semantic scene knowledge [2], and building reusable semantic representations from egocentric observations [3]. A brief discussion would help better position the proposed lifelong scene-memory formulation and clarify its differences from related navigation and mapping approaches.
[1] Kunyang Lin, Peihao Chen, Diwei Huang, Thomas H. Li, Mingkui Tan, and Chuang Gan. Learning Vision-and-Language Navigation from YouTube Videos. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8317–8326, 2023. Open Access
[2] Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman. PONI: Potential Functions for ObjectGoal Navigation with Interaction-Free Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18890–18900, 2022. Open Access
[3] Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, and Dhruv Batra. Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views. Proceedings of the AAAI Conference on Artificial Intelligence, 35(2):964–972, 2021.
6 comments
[Question 1] Generalization to Scenes Outside the Development Set
Thank you for raising this question. We used 10 lifelong navigation sequences from 9 scenes for method development and hyperparameter selection, covering 82 subtasks. We split LiSoNav-Eval at the scene level into val-seen and val-unseen. Val-seen contains different sequences from the same 9 scenes, excluding all 10 development sequences. Val-unseen contains the remaining 19 scenes.
**
Performance on val-seen**\ (9 scenes, 416 subtasks, including 178 relocated-target subtasks).
| Method | SR ↑ | SPL ↑ | HR | Relocated-target SR ↑ |
|---|---|---|---|---|
| VLFM | 14.7% | 10.5% | 32.0% | 14.0% |
| AstraNav | 13.0% | 9.9% | 22.7% | 9.6% |
| DualMap | 15.9% | 11.2% | 39.8% | 11.2% |
| DynaMem | 17.5% | 13.2% | 42.4% | 11.8% |
| DynaMem Oracle | 22.4% | 16.6% | 42.6% | 15.2% |
| 3D-Mem | 24.3% | 17.1% | 35.2% | 19.1% |
| 3D-Mem Oracle | 32.0% | 19.3% | 55.1% | 19.7% |
| IVAM-Nav (Ours) | 34.4% | 19.3% | 78.8% | 32.0% |
| IVAM-Nav (Ours) | 37.0% | 22.2% | 82.2% | 32.0% |
**
Performance on val-unseen** .
| Method | SR ↑ | SPL ↑ | HR | Relocated-target SR ↑ |
|---|---|---|---|---|
| VLFM | 11.0% | 7.9% | 25.2% | 10.6% |
| AstraNav | 11.7% | 9.2% | 23.4% | 9.0% |
| DualMap | 12.9% | 9.6% | 32.7% | 9.8% |
| DynaMem | 18.3% | 12.5% | 35.7% | 16.3% |
| DynaMem Oracle | 18.9% | 13.3% | 38.0% | 16.1% |
| 3D-Mem | 20.7% | 14.7% | 33.8% | 16.3% |
| 3D-Mem Oracle | 25.2% | 17.8% | 45.9% | 19.1% |
| IVAM-Nav (Ours) | 32.8% | 17.9% | 76.6% | 29.7% |
| IVAM-Nav (Ours) | 35.9% | 20.8% | 78.2% | 29.2% |
On both val-seen and val-unseen, IVAM-Nav achieves substantial gains over the strongest non-oracle baseline, 3D-Mem, in overall SR and relocated-target SR, while also achieving higher SPL. These results indicate that IVAM-Nav generalizes well to scenes not used during development. We also observe that SR is lower on val-unseen for almost all methods, and SPL is lower for all methods, indicating that unseen scenes are generally more challenging. We have clarified the scene-level development/evaluation split and included the split-specific results in the Appendix.
[Weakness 2 / Question 2] Runtime, Memory Use, and Scaling Across Episodes
Thank you for these helpful questions. We compare IVAM-Nav with 3D-Mem on the full LiSoNav-Eval benchmark. We use the official implementation of 3D-Mem. Both methods use the same hardware: an NVIDIA L20 GPU and Intel Xeon Platinum 8365A CPUs. All MLLM requests use the Qwen3-VL-Plus API through Alibaba Cloud. We measure API waiting time as the time spent waiting for MLLM API responses and wall-clock runtime as the end-to-end time to complete a subtask, including API waiting, local computation and planning, and simulator navigation. We also measure memory usage and disk storage to examine memory growth across episodes. Memory usage mainly covers navigation maps and retrieval-related data, while observation images are stored on disk.
**
Average computational cost per subtask on the full benchmark.**
| Method | Output tokens | MLLM calls | Total API waiting time | Wall-clock runtime | Memory usage | Disk storage |
|---|---|---|---|---|---|---|
| 3D-Mem | 3.04k | 22.22 | 71.41 | 180.21 | 23.30 | 37.37 |
| IVAM-Nav (Ours) | 1.45k | 28.06 | 30.30 | 95.92 | 22.93 | 36.41 |
Overall computational cost. IVAM-Nav generates fewer output tokens per subtask than 3D-Mem 1\.45k vs\. 3\.04k, as it does not require lengthy chain-of-thought rationales. It makes more MLLM calls 28\.06 vs\. 22\.22 because it separately requests per-view descriptions and target verification. Since IVAM-Nav selects multiple complementary views at once, these independent requests can be submitted in parallel. Fewer output tokens reduce response-generation time and thus API waiting time, while parallel execution further reduces waiting time by overlapping independent requests. Together, these factors result in 57.6% less API waiting time and 46.8% shorter wall-clock runtime than 3D-Mem. Meanwhile, IVAM-Nav has comparable memory usage and disk storage to 3D-Mem 22\.93 vs\. 23\.30 MB and 36\.41 vs\. 37\.37 MB, respectively.
Memory growth and runtime across episodes. To examine how memory size, retrieval load, and runtime evolve across episodes, we collect per-subtask statistics for each episode and break down API waiting time across three core stages. IVAM-Nav first revisits historical viewpoints for memory revalidation and target verification. If the target is not confirmed, it proceeds to active exploration to decide where to explore, followed by multi-view inspection of selected supporting surfaces. Table cHVx-6 reports the episode-wise statistics, including API waiting time for each stage. Other API operations include auxiliary steps such as target-description normalization and rephrasing.
**
Episode-wise memory, retrieval, and runtime statistics averaged per subtask.** Each LiSoNav-Eval sequence contains three or four episodes, with a varying number of subtasks per episode. The weighted average is computed over all subtasks.
| Episode | Stored viewpoints | Memory usage | Disk storage | Filtered candidates | MLLM calls | Total API waiting time | API waiting time for historical revisits | API waiting time for active exploration | API waiting time for multi-view inspection | Other API waiting time | Wall-clock runtime |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 7.66 | 22.11 | 14.67 | 3.68 | 31.15 | 42.56 | 3.30 | 23.33 | 14.40 | 1.53 | 148.31 |
| 2 | 22.22 | 22.39 | 39.30 | 8.85 | 26.15 | 27.40 | 7.89 | 11.22 | 6.96 | 1.33 | 85.47 |
| 3 | 32.50 | 23.75 | 47.13 | 11.03 | 27.26 | 24.80 | 10.78 | 8.28 | 4.34 | 1.41 | 70.53 |
| 4 | 45.58 | 24.03 | 53.23 | 11.17 | 27.30 | 22.35 | 11.76 | 6.20 | 2.92 | 1.48 | 61.93 |
| Weighted average | 24.19 | 22.93 | 36.41 | 8.32 | 28.06 | 30.30 | 7.94 | 13.15 | 7.78 | 1.43 | 95.92 |
As episodes progress, the number of stored viewpoints and disk usage increase, while memory usage remains relatively stable. Despite the growing navigation memory, the number of filtered candidates levels off in later episodes, and MLLM calls per subtask remain broadly stable. With more historical information available, the agent relies more on historical revisits and less on active exploration and multi-view inspection. Although API waiting time for historical revisits increases, the reductions in exploration and multi-view inspection time outweigh this increase, leading to lower overall API waiting time and wall-clock runtime.
Thank you for highlighting the task setting and viewpoint-anchored memory. Below, we report results on furniture relocation and scenes held out from development, analyze runtime and memory over longer sequences, test alternative MLLMs and reworded internal prompts, and discuss the related work you suggested. We would appreciate it if you could let us know whether your concerns are addressed by our response.
[Weakness 1] Concerns Regarding Limited Types of Environmental Changes
Thank you for raising this concern. The original benchmark considers small-object relocation while keeping supporting surfaces and the rest of the background scene unchanged. In real homes, furniture can move, localization can drift, and lighting or clutter can change.
To evaluate furniture movement and changes to supporting surfaces, we constructed an additional subset with movable supporting furniture. Specifically, we selected movable furniture from HSSD, such as stools, and placed small objects on them. Between episodes, these furniture items can be relocated together with the objects placed on them; small objects can also be moved onto or off these items. The subset covers the same 28 scenes and contains 226 subtasks, including 112 relocated-target subtasks. Of these 112 subtasks, 66 589˙ involve movable supporting furniture. We evaluated IVAM-Nav and 3D-Mem on this subset.
**
TablecHVx−1Performance on the separate extension subset with changing supporting surfaces.** The final column reports SR on relocated-target subtasks involving the added movable supporting furniture.
As shown in Table cHVx-1, IVAM-Nav achieves higher overall SR and SPL than 3D-Mem, and also performs better on relocated-target subtasks involving movable supporting furniture. Moving a support can invalidate both the remembered target location and its supporting-surface information. IVAM-Nav detects such inconsistencies through historical-viewpoint revisitation and updates the outdated local memory to prevent it from misleading subsequent navigation. It then continues exploring to locate the support and target at their new position.
Compared with the results on the original benchmark, overall SR decreases from 21.9% to 19.0% for 3D-Mem and from 33.4% to 32.3% for IVAM-Nav, indicating that the extension presents a more challenging setting. We have added the construction details and evaluation results of this extension in the Appendix. We will release the complete LiSoNav-Eval dataset, including both the original benchmark and the extension.
We further evaluate the effect of localization drift on historical-viewpoint revisitation using the 245-subtask analysis subset for efficient evaluation. This evaluation is conducted without target reference images. At each revisit, we randomly sample horizontal position and heading offsets uniformly within the specified bounds relative to the stored pose.
**
TablecHVx−2Sensitivity to localization errors during historical-viewpoint revisitation.**
As the localization perturbation increases, revalidation accuracy decreases from 77.5% to 73.2%, as position and heading errors make the current observation harder to match with the stored view. However, this decrease has a limited impact on navigation success: under the largest perturbation, SR decreases only from 31.4% to 30.2%. This is because a failed revisit does not end the search. IVAM-Nav can check other historical viewpoints or continue exploring to locate the target. This mechanism helps mitigate the impact of localization errors on final navigation performance.
We appreciate the reviewer's understanding of this limitation. Broader environmental changes, including lighting and clutter variations, remain important directions for future work.