acceptodds
Under review as a conference paper at ICLR 2027

Lifelong Small-Object Navigation in Changing Object Layouts: A Benchmark and Method

Abstract

Household robots need to continually navigate to different objects in the same environment, many of which are small and portable, such as tools and toys. Their small visual footprint and frequent occlusion make reliable observation difficult, and they may be moved by people without the robot observing the changes. We formulate this challenging task as Lifelong Small-object Navigation in Changing Object Layouts (LiSoNav-COL). Agents must seek suitable viewpoints for reliable observation, accumulate and reuse scene knowledge to efficiently locate subsequent targets, and update outdated memory after object relocation. To eliminate the need for prior scene scanning, we also require agents to start navigation with empty scene memory. Although practical, this task still lacks benchmarks designed around its defining assumptions. To bridge this gap, we introduce LiSoNav-Eval, a dedicated benchmark spanning 28 indoor scenes with 45 small-object categories. Its lifelong navigation sequences include both unchanged and relocated targets to evaluate memory reuse and adaptation to object relocation. To address this challenging task, we propose a navigation method based on multi-view Inspection with Viewpoint-Anchored Memory, dubbed IVAM-Nav. IVAM-Nav actively observes supporting surfaces from complementary viewpoints for reliable small-object perception and anchors the resulting memory to their observation viewpoints, supporting relational memory reuse and revalidation under similar viewing conditions. Extensive experiments on LiSoNav-Eval demonstrate favorable performance of IVAM-Nav against representative methods. Benchmark analyses also show that smaller objects, larger environments, and longer relocation distances pose greater challenges. We will release the dataset and code.

Primary area:
datasets and benchmarks
Keywords:
Lifelong Navigation, Small-Object Navigation, Changing Object Layouts, Object Relocation, Scene Memory, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cHVxOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers a lifelong object navigation setting where the agent repeatedly searches for small household objects in the same environment, while some objects may be moved between episodes without the agent observing the change. The authors name this setting LiSoNav-COL and introduce LiSoNav-Eval, a benchmark with 28 scenes, 1,199 subtasks, and 45 object categories. Around 45% of the subtasks involve relocated targets.

The proposed method, IVAM-Nav, builds a persistent graph memory around historical viewpoints. During exploration, the agent actively observes supporting surfaces from several viewpoints and stores object/surface information together with the corresponding camera poses. For later tasks, it retrieves relevant historical viewpoints, revisits them, checks whether the stored information is still valid, and updates stale local memory when necessary.

The experiments compare IVAM-Nav with several navigation and scene-memory baselines. IVAM-Nav reaches 33.4% overall SR, compared with 27.5% for the strongest reported oracle baseline, and achieves 30.5% SR on relocated targets. The ablations further show that complementary viewpoints, graph aggregation, historical-viewpoint revisit, and memory updates all contribute to the final performance.

Critical strengths
  1. The problem setting is well motivated and not fully covered by existing benchmarks.

The paper combines several issues that matter in practice: small objects are hard to observe, the agent starts without a prebuilt scene memory, past experience should help future tasks, and that memory can become stale when objects are moved. I think this is a meaningful setting for lifelong embodied navigation.

  1. The method is simple enough to understand, and the main design choices are backed by experiments.

The strongest part of IVAM-Nav, in my view, is the use of historical viewpoints as memory anchors. Instead of only storing that an object was seen near a certain place, the method also keeps track of where the observation came from and later returns to a similar viewpoint for revalidation. The gains are also fairly substantial. IVAM-Nav outperforms the reported baselines in overall SR and performs especially well on relocated targets. The ablations and the separate revalidation experiment make the improvement easier to attribute to the proposed components rather than only to the overall system complexity.

Critical weaknesses
  1. The type of environmental change considered in the benchmark is still fairly limited.

The benchmark changes the locations of small objects, while the supporting surfaces and the rest of the background scene remain stable. The authors also mention this limitation themselves.
This matters because the viewpoint-anchored memory assumes that returning to a historical viewpoint is still meaningful. In a real home, furniture can move, supporting surfaces can change, localization can drift, and lighting or clutter can change significantly. It is therefore not yet clear how well the same mechanism would work under broader scene changes.

I do not see this as a fatal issue for the current paper, but it makes the real-world motivation somewhat stronger than the current evaluation can fully support.

  1. The paper does not give enough information about the computational cost and long-term scaling of the method.

IVAM-Nav uses active multi-view inspection, repeated historical revisits, and MLLM calls at several stages. SPL captures path efficiency, but it does not tell us how expensive the system is in terms of inference time, number of MLLM calls, or memory growth. This is particularly relevant because the paper presents the problem as lifelong navigation, while the benchmark sequences only contain three or four episodes. The authors show that the number of retrieved historical candidates grows across episodes, and they also acknowledge that multi-view observation introduces additional navigation cost.
I would have liked to see at least a basic runtime and memory analysis, or an experiment showing how the method behaves over much longer sequences.

Questions for authors
  1. Is there a fully held-out test split?

The appendix describes the 1,199-subtask set as the full LiSoNav-Eval validation set, and the method-analysis subset is sampled from it.
Could the authors clarify whether the reported 28 scenes were also used during method development or hyperparameter selection? Is there a separate held-out test split that was not used during development?

This would affect my confidence in how much of the reported gain generalizes beyond this particular benchmark instance.

  1. How expensive is IVAM-Nav in practice?

Could the authors report the average number of MLLM calls, wall-clock runtime, and memory size per subtask or per episode? It would also be useful to show how these quantities grow as the number of episodes increases. Since the method repeatedly revisits historical viewpoints and actively collects multiple views, this seems important for evaluating the practicality of the “lifelong” setting.

  1. How sensitive is the method to the particular MLLM used?

The benchmark construction uses Qwen-VL-Max for annotation/verification, while IVAM-Nav uses Qwen3-VL for several semantic and decision-making steps.
Have the authors tested another MLLM, or at least a different set of independently written target/context descriptions? I would be interested in knowing whether the gain is stable across model choices and language styles.

  1. The related work could be expanded a little.

There is prior work studying how navigation agents acquire and exploit environmental knowledge from different perspectives, including learning navigation and layout priors from large-scale real-world videos [1], object-centric search based on semantic scene knowledge [2], and building reusable semantic representations from egocentric observations [3]. A brief discussion would help better position the proposed lifelong scene-memory formulation and clarify its differences from related navigation and mapping approaches.

[1] Kunyang Lin, Peihao Chen, Diwei Huang, Thomas H. Li, Mingkui Tan, and Chuang Gan. Learning Vision-and-Language Navigation from YouTube Videos. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8317–8326, 2023. Open Access

[2] Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman. PONI: Potential Functions for ObjectGoal Navigation with Interaction-Free Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18890–18900, 2022. Open Access

[3] Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, and Dhruv Batra. Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views. Proceedings of the AAAI Conference on Artificial Intelligence, 35(2):964–972, 2021.

6 comments

Thank you for highlighting the task setting and viewpoint-anchored memory. Below, we report results on furniture relocation and scenes held out from development, analyze runtime and memory over longer sequences, test alternative MLLMs and reworded internal prompts, and discuss the related work you suggested. We would appreciate it if you could let us know whether your concerns are addressed by our response.

[Weakness 1] Concerns Regarding Limited Types of Environmental Changes

Thank you for raising this concern. The original benchmark considers small-object relocation while keeping supporting surfaces and the rest of the background scene unchanged. In real homes, furniture can move, localization can drift, and lighting or clutter can change.

To evaluate furniture movement and changes to supporting surfaces, we constructed an additional subset with movable supporting furniture. Specifically, we selected movable furniture from HSSD, such as stools, and placed small objects on them. Between episodes, these furniture items can be relocated together with the objects placed on them; small objects can also be moved onto or off these items. The subset covers the same 28 scenes and contains 226 subtasks, including 112 relocated-target subtasks. Of these 112 subtasks, 66 589˙58\.9% involve movable supporting furniture. We evaluated IVAM-Nav and 3D-Mem on this subset.

**

TablecHVx−1Table cHVx-1

Performance on the separate extension subset with changing supporting surfaces.** The final column reports SR on relocated-target subtasks involving the added movable supporting furniture.

MethodSR ↑SPL ↑Relocated-target SR on changing supporting surfaces ↑
3D-Mem19.0%14.4%16.7%
IVAM-Nav (Ours)32.3%16.7%33.3%

As shown in Table cHVx-1, IVAM-Nav achieves higher overall SR and SPL than 3D-Mem, and also performs better on relocated-target subtasks involving movable supporting furniture. Moving a support can invalidate both the remembered target location and its supporting-surface information. IVAM-Nav detects such inconsistencies through historical-viewpoint revisitation and updates the outdated local memory to prevent it from misleading subsequent navigation. It then continues exploring to locate the support and target at their new position.

Compared with the results on the original benchmark, overall SR decreases from 21.9% to 19.0% for 3D-Mem and from 33.4% to 32.3% for IVAM-Nav, indicating that the extension presents a more challenging setting. We have added the construction details and evaluation results of this extension in the Appendix. We will release the complete LiSoNav-Eval dataset, including both the original benchmark and the extension.

We further evaluate the effect of localization drift on historical-viewpoint revisitation using the 245-subtask analysis subset for efficient evaluation. This evaluation is conducted without target reference images. At each revisit, we randomly sample horizontal position and heading offsets uniformly within the specified bounds relative to the stored pose.

**

TablecHVx−2Table cHVx-2

Sensitivity to localization errors during historical-viewpoint revisitation.**

Position / heading perturbation boundsSR ↑SPL ↑Relocated-target SR ↑Revalidation accuracy ↑
No perturbation31.4%18.1%30.8%77.5%
5 cm / ±5°30.6%17.5%29.9%76.3%
10 cm / ±5°31.4%16.8%31.6%74.7%
10 cm / ±10°31.0%17.1%29.1%73.9%
20 cm / ±10°30.2%16.6%29.1%73.2%

As the localization perturbation increases, revalidation accuracy decreases from 77.5% to 73.2%, as position and heading errors make the current observation harder to match with the stored view. However, this decrease has a limited impact on navigation success: under the largest perturbation, SR decreases only from 31.4% to 30.2%. This is because a failed revisit does not end the search. IVAM-Nav can check other historical viewpoints or continue exploring to locate the target. This mechanism helps mitigate the impact of localization errors on final navigation performance.

We appreciate the reviewer's understanding of this limitation. Broader environmental changes, including lighting and clutter variations, remain important directions for future work.

[Question 1] Generalization to Scenes Outside the Development Set

Thank you for raising this question. We used 10 lifelong navigation sequences from 9 scenes for method development and hyperparameter selection, covering 82 subtasks. We split LiSoNav-Eval at the scene level into val-seen and val-unseen. Val-seen contains different sequences from the same 9 scenes, excluding all 10 development sequences. Val-unseen contains the remaining 19 scenes.

**

TablecHVx−3Table cHVx-3

Performance on val-seen**\ (9 scenes, 416 subtasks, including 178 relocated-target subtasks).

MethodSR ↑SPL ↑HRRelocated-target SR ↑
VLFM14.7%10.5%32.0%14.0%
AstraNav13.0%9.9%22.7%9.6%
DualMap15.9%11.2%39.8%11.2%
DynaMem17.5%13.2%42.4%11.8%
DynaMem Oracle22.4%16.6%42.6%15.2%
3D-Mem24.3%17.1%35.2%19.1%
3D-Mem Oracle32.0%19.3%55.1%19.7%
IVAM-Nav (Ours)34.4%19.3%78.8%32.0%
IVAM-Nav w/referenceimagew/ reference image (Ours)37.0%22.2%82.2%32.0%

**

TablecHVx−4Table cHVx-4

Performance on val-unseen** 19scenes,783subtasks,including367relocated−targetsubtasks19 scenes, 783 subtasks, including 367 relocated-target subtasks.

MethodSR ↑SPL ↑HRRelocated-target SR ↑
VLFM11.0%7.9%25.2%10.6%
AstraNav11.7%9.2%23.4%9.0%
DualMap12.9%9.6%32.7%9.8%
DynaMem18.3%12.5%35.7%16.3%
DynaMem Oracle18.9%13.3%38.0%16.1%
3D-Mem20.7%14.7%33.8%16.3%
3D-Mem Oracle25.2%17.8%45.9%19.1%
IVAM-Nav (Ours)32.8%17.9%76.6%29.7%
IVAM-Nav w/referenceimagew/ reference image (Ours)35.9%20.8%78.2%29.2%

On both val-seen and val-unseen, IVAM-Nav achieves substantial gains over the strongest non-oracle baseline, 3D-Mem, in overall SR and relocated-target SR, while also achieving higher SPL. These results indicate that IVAM-Nav generalizes well to scenes not used during development. We also observe that SR is lower on val-unseen for almost all methods, and SPL is lower for all methods, indicating that unseen scenes are generally more challenging. We have clarified the scene-level development/evaluation split and included the split-specific results in the Appendix.

[Weakness 2 / Question 2] Runtime, Memory Use, and Scaling Across Episodes

Thank you for these helpful questions. We compare IVAM-Nav with 3D-Mem on the full LiSoNav-Eval benchmark. We use the official implementation of 3D-Mem. Both methods use the same hardware: an NVIDIA L20 GPU 45GiB45 GiB and Intel Xeon Platinum 8365A CPUs. All MLLM requests use the Qwen3-VL-Plus API through Alibaba Cloud. We measure API waiting time as the time spent waiting for MLLM API responses and wall-clock runtime as the end-to-end time to complete a subtask, including API waiting, local computation and planning, and simulator navigation. We also measure memory usage and disk storage to examine memory growth across episodes. Memory usage mainly covers navigation maps and retrieval-related data, while observation images are stored on disk.

**

TablecHVx−5Table cHVx-5

Average computational cost per subtask on the full benchmark.**

MethodOutput tokensMLLM callsTotal API waiting time ssWall-clock runtime ssMemory usage MBMBDisk storage MBMB
3D-Mem3.04k22.2271.41180.2123.3037.37
IVAM-Nav (Ours)1.45k28.0630.3095.9222.9336.41

Overall computational cost. IVAM-Nav generates fewer output tokens per subtask than 3D-Mem 1\.45k vs\. 3\.04k, as it does not require lengthy chain-of-thought rationales. It makes more MLLM calls 28\.06 vs\. 22\.22 because it separately requests per-view descriptions and target verification. Since IVAM-Nav selects multiple complementary views at once, these independent requests can be submitted in parallel. Fewer output tokens reduce response-generation time and thus API waiting time, while parallel execution further reduces waiting time by overlapping independent requests. Together, these factors result in 57.6% less API waiting time and 46.8% shorter wall-clock runtime than 3D-Mem. Meanwhile, IVAM-Nav has comparable memory usage and disk storage to 3D-Mem 22\.93 vs\. 23\.30 MB and 36\.41 vs\. 37\.37 MB, respectively.

Memory growth and runtime across episodes. To examine how memory size, retrieval load, and runtime evolve across episodes, we collect per-subtask statistics for each episode and break down API waiting time across three core stages. IVAM-Nav first revisits historical viewpoints for memory revalidation and target verification. If the target is not confirmed, it proceeds to active exploration to decide where to explore, followed by multi-view inspection of selected supporting surfaces. Table cHVx-6 reports the episode-wise statistics, including API waiting time for each stage. Other API operations include auxiliary steps such as target-description normalization and rephrasing.

**

TablecHVx−6Table cHVx-6

Episode-wise memory, retrieval, and runtime statistics averaged per subtask.** Each LiSoNav-Eval sequence contains three or four episodes, with a varying number of subtasks per episode. The weighted average is computed over all subtasks.

EpisodeStored viewpointsMemory usage MBMBDisk storage MBMBFiltered candidatesMLLM callsTotal API waiting time ssAPI waiting time for historical revisits ssAPI waiting time for active exploration ssAPI waiting time for multi-view inspection ssOther API waiting time ssWall-clock runtime ss
17.6622.1114.673.6831.1542.563.3023.3314.401.53148.31
222.2222.3939.308.8526.1527.407.8911.226.961.3385.47
332.5023.7547.1311.0327.2624.8010.788.284.341.4170.53
445.5824.0353.2311.1727.3022.3511.766.202.921.4861.93
Weighted average24.1922.9336.418.3228.0630.307.9413.157.781.4395.92

As episodes progress, the number of stored viewpoints and disk usage increase, while memory usage remains relatively stable. Despite the growing navigation memory, the number of filtered candidates levels off in later episodes, and MLLM calls per subtask remain broadly stable. With more historical information available, the agent relies more on historical revisits and less on active exploration and multi-view inspection. Although API waiting time for historical revisits increases, the reductions in exploration and multi-view inspection time outweigh this increase, leading to lower overall API waiting time and wall-clock runtime.

Reviewer_W9VWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LiSoNav-COL and LiSoNav-Eval for lifelong navigation to small portable objects, where an agent starts with empty memory, repeatedly operates in the same environment, and must handle unobserved object relocations between episodes. It further proposes IVAM-Nav, which combines close-range complementary multi-view inspection, viewpoint-anchored graph memory, relational retrieval, and pose-matched memory revalidation and local updating. On 28 scenes, 45 object categories, and 1,199 subtasks, IVAM-Nav achieves 33.4% SR / 18.4% SPL and 30.5% relocated-target SR, outperforming the evaluated scene-memory baselines.

Critical strengths
  1. Meaningful benchmark setting. LiSoNav-Eval brings together small-object perception, online memory construction from an empty state, repeated tasks, and hidden inter-episode relocation. The mixture of unchanged and relocated targets usefully tests both memory reuse and adaptation to stale information, rather than assuming that all prior memory is either valid or invalid.
Critical weaknesses
  1. The source of the small-object improvement remains causally unclear. The method simultaneously introduces close-range observation, multiple and complementary viewpoints, depth masking, contextual cues, graph aggregation, and memory revalidation. Although the full method performs well on objects below 1 L, the paper does not report small-object recognition or memory-construction accuracy as a function of size, distance, viewpoint count, or sensing budget. Controlled single-view versus matched-budget multi-view experiments and perception-level recall would clarify what specifically enables better small-object discovery.

  2. The novelty is primarily in the integration of existing challenges. Empty-memory lifelong acquisition, dynamic memory updates, multi-view scene memory, sequential goals, and small-object navigation each have precedents. The distinction from OpenIN is meaningful—especially empty initialization, repeated hidden relocation, and the mixture of unchanged and relocated targets—but the contribution should be framed as a new intersection of these challenges rather than the first dynamic or lifelong navigation setting. Moreover, the benchmark does not demonstrate learning household-specific placement habits; relocations are randomized over semantically compatible surfaces rather than generated from persistent environment-specific placement distributions.

  3. Stale memory is mitigated only after incurring a revisit cost, and physical validation is absent. IVAM-Nav initially follows historical retrieval and must travel to an old viewpoint before determining that the memory is stale; the paper does not quantify the distance or time wasted on invalidated locations, stale candidates checked, or repeated invalid-location revisits. The lack of real-robot experiments also leaves open how reliably small-object inspection and pose-matched revalidation work under localization, perception, and actuation noise.

Questions for authors

The benchmark is proposed from the challenges like relocation and small objects but it seems that IVAM-Nav does not directly address these challenges:

  • which components allows for achieving better performance in finding small objects?
  • which components allows for achieving better performance in finding relocated objects?
3 comments

Thank you for recognizing how unchanged and relocated targets test memory reuse and adaptation to stale information. Below, we report controlled experiments on inspection distance and viewpoint selection for small-object discovery, and ablations for finding relocated targets. We clarify the task's contribution. We also quantify the frequency and travel cost of revisiting stale viewpoints and assess sensitivity to revisit pose errors. We would appreciate it if you could let us know whether your concerns are addressed by our response.

[Weakness 1 / Question 1] Components That Improve Small-Object Discovery

Thank you for raising this concern. We conduct controlled observation ablations to examine how inspection distance and viewpoint selection affect small-object discovery. We measure memory-construction recall for two object-volume groups under different recognition-input budgets. All variants use the same MLLM and evaluation episodes.

**

TableW9VW−1Table W9VW-1

Effects of inspection distance and viewpoint selection on memory-construction recall.**

RowSettingProcedureMean number of 640 × 640 images processed per supporting surface ↓Overall recall % ↑Recall % Volume<1LVolume < 1 L ↑Recall % (Volume ≥ 1 L) ↑
1Increased inspection distanceUse our selection method but place each selected viewpoint 0.5 m farther from the supporting surface.2.340.535.751.4
2Single-view cropsCapture one 1280 × 1280 image from a single viewpoint and divide it into four 640 × 640 crops.4.050.25246.2
3Four complementary viewpointsGreedily select four complementary viewpoints and capture one 640 × 640 image at each.4.06460.971.0
4Count-matched random viewpointsRandomly select the same number of viewpoints as in Ours.1.945.242.152.3
5OursGreedily add complementary viewpoints until the coverage threshold is reached.1.957.152.368.2

Metrics. All evaluated objects belong to the small-object set defined in our original evaluation. We divide this set into two volume groups: below 1 L and at least 1 L. Recall is the percentage of evaluated objects identifiable from their stored descriptions, as judged by GPT-6.1 using the target description and reference image. Object volume is estimated from its bounding-box volume in liters LL. The image-count column reports the mean number of 640 × 640 images processed by the recognition model per supporting surface.

Table W9VW-1 reveals several factors affecting memory-construction recall:

  • Inspection distance matters. Moving viewpoints 0.5 m farther from the supporting surface Row1Row 1 reduces recall for objects with volume below 1 L from 52.3% (Ours, Row 5) to 35.7%, despite processing more images per surface 2\.3 vs\. 1\.9.

  • Complementary viewpoints improve recall under a matched input budget. With four 640 × 640 images per surface, four complementary viewpoints Row3Row 3 achieve 60.9% recall for objects below 1 L, compared with 52.0% for four crops from a single viewpoint Row2Row 2.

  • Greedy viewpoint selection outperforms random selection. With the same average image count of 1.9 per surface, greedy complementary selection Row5Row 5 achieves 52.3% recall for objects below 1 L, compared with 42.1% for random selection Row4Row 4.

  • Object volume affects recall. Within the evaluated small-object set, objects below 1 L have lower recall than those at least 1 L in four of the five settings. With ours, the respective recall rates are 52.3% and 68.2%, indicating that objects with volumes below 1 L remain more difficult to capture in memory.

  • Our method balances recall and recognition-input budget. Compared with always selecting four complementary viewpoints Row3Row 3, Ours uses 52.5% fewer images per surface 1\.9 vs\. 4\.0, with recall for objects below 1 L decreasing from 60.9% to 52.3%. Compared with four crops from a single viewpoint Row2Row 2, Ours uses fewer images while achieving similar recall for objects below 1 L 523˙52\.3% vs\. 52\.0% and higher overall recall 571˙57\.1% vs\. 50\.2%.

[Weakness 2] Clarification of Novelty

Thank you for this helpful comment. Our contribution lies in bringing established challenges together in a unified setting, rather than introducing each challenge individually. LiSoNav-COL combines initially empty scene memory, repeated small-object searches, and unobserved object relocations in a practical lifelong navigation setting. This formulation allows us to study how agents acquire, reuse, and update scene-specific knowledge over time. We appreciate the reviewer's recognition that this integrated task formulation makes a meaningful distinction from OpenIN. We apologize that our original contribution statement did not make this positioning sufficiently clear. We have revised the contribution statement in the Introduction of the revised manuscript to better emphasize this integrated task formulation.

[Weakness 2] Household-Specific Placement Habits

We sincerely appreciate this suggestion. LiSoNav-Eval currently samples object relocations among semantically compatible supporting surfaces, rather than modeling persistent, household-specific placement patterns. Modeling such patterns would require longitudinal observations from individual households to capture recurring object placements across rooms and supporting surfaces. It would also require distinguishing habitual placements from incidental relocations and developing methods to learn and use these patterns during subsequent searches. Collecting and annotating such data across households, together with developing the corresponding learning and evaluation framework, would require substantial additional effort. Given this scope, we consider it better suited to a dedicated follow-up study and have identified it as a direction for future work in the revised manuscript.

[Weakness 3] Revisit Cost of Stale Memory

Thank you for highlighting this cost. When an object has been observed before, returning to its last known location is a natural first step. This allows the agent to quickly check whether the object is still there. People often follow the same approach when looking for household items. If the object has moved, the revisit incurs additional travel before the agent can redirect its search. We quantify this cost over all navigation subtasks in Table W9VW-2.

**

TableW9VW−2Table W9VW-2

Stale-viewpoint revisit costs.**

MethodMean stale viewpoints revisited per subtaskMean travel distance to stale viewpoints per subtask mm
IVAM-Nav (Ours)1.815.47

On average, IVAM-Nav revisits 1.81 stale historical viewpoints per subtask, requiring 5.47 m of travel. Please note that IVAM-Nav checks each historical viewpoint at most once per subtask, preventing repeated visits to the same stale location.

Although IVAM-Nav revisits only 1.81 stale viewpoints per subtask on average, this number could be further reduced through more selective memory maintenance. As discussed in Appendix E, future work could exploit observations acquired during normal navigation to identify memories that require revalidation and balance the expected benefits against navigation costs. We have clarified this revisit policy and its associated cost in the Appendix.

[Weakness 3] Physical Evaluation and Sensitivity to Viewpoint Revisitation Errors

Thank you for highlighting the need for real-robot experiments. Such experiments are important for evaluating how reliably small-object inspection and pose-matched revalidation work under localization, perception, and actuation noise. However, we currently lack a suitable navigation robot and an appropriate real-world test environment, making physical evaluation infeasible in this work.

Although we cannot conduct real-robot experiments, we assess the sensitivity of pose-matched revalidation to revisit-pose errors through simulated perturbations. Specifically, whenever IVAM-Nav revisits a historical viewpoint, we randomly sample horizontal-position and heading offsets uniformly within the specified bounds relative to the stored pose. The agent then acquires a new observation from the perturbed pose. We conduct this experiment on the 245-subtask analysis subset from Sec. 5.3 without target reference images. Revalidation accuracy measures how accurately the agent identifies consistency between the current observation and the stored view.

**

TableW9VW−3Table W9VW-3

Sensitivity of viewpoint-anchored revalidation to position and heading errors.**

Position / heading perturbation boundsSR ↑SPL ↑Relocated-target SR ↑Revalidation accuracy ↑
No perturbation31.4%18.1%30.8%77.5%
5 cm / ±5°30.6%17.5%29.9%76.3%
10 cm / ±5°31.4%16.8%31.6%74.7%
10 cm / ±10°31.0%17.1%29.1%73.9%
20 cm / ±10°30.2%16.6%29.1%73.2%

Revalidation accuracy decreases from 77.5% to 73.2% as larger position and heading errors make the current observation harder to match with the stored view. However, this decrease has a limited impact on navigation success, with SR dropping only from 31.4% to 30.2% under the largest perturbation. This is because even when a revisit fails to confirm the target, IVAM-Nav can check other historical viewpoints or continue exploring to locate it. This mechanism helps mitigate the impact of revisitation errors on final navigation performance.

This simulation provides a controlled sensitivity analysis of pose-matched revalidation. Full evaluation under real-world robot noise remains future work. We have included this analysis in the Appendix.

[Question 2] Components for Finding Relocated Objects

Thank you for this question. We examine three components that support finding relocated objects. Graph aggregation uses object and supporting-surface relations to retrieve relevant historical regions. Memory update corrects outdated object records after relocation to prevent stale information from misleading subsequent navigation. Complementary viewpoints improve supporting-surface coverage during exploration, helping the agent find objects at their new locations when historical retrieval fails. We ablate each component and report SR and SPL on relocated-target subtasks. All variants use the same analysis subset and target reference images as in Sec. 5.3.

**

TableW9VW−4Table W9VW-4

Component ablations for navigation to relocated targets.**

SettingAblation descriptionRelocated-target SR % ↑Relocated-target SPL % ↑
w/o graph aggregationIndependently matches memory candidates with the target object without graph aggregation around historical viewpoints.22.2211.16
w/o memory updateLeaves inconsistent historical memory unchanged and continues to use it for subsequent navigation decisions.16.249.95
w/o complementary viewpointsRetains only one viewpoint near each supporting surface during memory construction.22.2211.4
FullAll components enabled.34.1918.73

The performance gaps between the full method and its ablations highlight the importance of all three components. Removing graph aggregation reduces SR and SPL by 11.97 and 7.57 percentage points, respectively, demonstrating the benefit of relational memory retrieval. Disabling memory update causes the largest drop, with SR and SPL decreasing by 17.95 and 8.78 points, respectively, highlighting its importance in correcting outdated information after relocation. Removing complementary viewpoints also reduces SR and SPL by 11.97 and 7.33 points, respectively, showing the benefit of broader supporting-surface coverage when searching for relocated objects.

Reviewer_j5QuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

changing object layouts, and LiSoNav-Eval, a benchmark containing 28 indoor scenes and 1,199 navigation subtasks. The benchmark requires agents to build scene memory from scratch, reuse previous observations, and adapt when objects are relocated between episodes.

The authors also propose IVAM-Nav, which combines active multi-view inspection, viewpoint-anchored graph memory, and historical memory revalidation. Experiments show improvements over several existing navigation methods, with an overall success rate of 33.4% and a relocated-target success rate of 30.5%.

Critical strengths
  1. Meaningful benchmark formulation.

The paper addresses a practical yet insufficiently studied setting that combines small-object perception, lifelong memory reuse, and object relocation. Unlike benchmarks focusing mainly on static environments or requiring preconstructed scene knowledge, LiSoNav-Eval evaluates memory acquisition and maintenance from an initially empty memory. The dataset construction and additional benchmark analyses make this a potentially useful resource for future research.

  1. Effective viewpoint-anchored memory design.

The proposed method connects complementary multi-view observations with viewpoint-anchored memory retrieval and revalidation. The experiments generally support the design choices. In particular, the ablations show that removing memory updates or complementary viewpoints substantially reduces navigation success. The additional revalidation analysis also supports revisiting historical viewpoints to identify changes in small-object configurations.

Critical weaknesses
  1. Simplified environmental changes and potentially informative target descriptions.

LiSoNav-Eval relocates small objects while keeping supporting surfaces and the background scene unchanged. Moreover, Appendix B.5 indicates that contextual descriptions are generated from the target's current placement after relocation. Although these descriptions do not directly reveal the target coordinates, they provide valid location-related cues that may simplify recovery from stale memory. The target-only experiments partially address this concern, but the benchmark's ability to represent less constrained real-world changes remains uncertain.

  1. Insufficient evaluation of computational and navigation overhead.

IVAM-Nav requires visiting multiple viewpoints and performing several MLLM-based operations for exploration, memory construction, revalidation, and target confirmation. However, the evaluation does not provide a detailed comparison of runtime, model calls, or computational cost. Since the reported performance gains may partly result from additional observations and inference, the practical efficiency of the proposed method remains unclear.

Questions for authors

1.Could the authors provide a more comprehensive evaluation without current-location contextual cues, especially for relocated targets? This would help clarify how much the benchmark and method rely on accurate post-relocation context.

2.What are the average runtime, MLLM call count, and navigation distance per subtask for IVAM-Nav compared with the strongest baselines? How much additional overhead is introduced by multi-view inspection and memory revalidation?

3.How sensitive is viewpoint-anchored revalidation to errors in robot localization and orientation? The current evaluation assumes that historical viewpoints can be revisited, but the impact of imperfect pose recovery is unclear.

4 comments

Thank you for highlighting the benchmark design and viewpoint-anchored memory. Below, we address your concerns about movable furniture, target-only instructions, computational and navigation costs, and sensitivity to pose errors. We would appreciate it if you could let us know whether your concerns are addressed by our response.

[Weakness 1] Concerns Regarding Simplified Environmental Changes

Thank you for highlighting this limitation. In household environments, supporting surfaces such as tables and beds are generally not moved frequently, although they may occasionally be relocated. Keeping the supporting surfaces and the background scene unchanged therefore limits the range of environmental changes represented in the original benchmark.

To address this limitation, we constructed an additional subset with movable supporting furniture. Specifically, we selected movable furniture from HSSD, such as stools, and placed small objects on them. Between episodes, these furniture items can be relocated together with the objects placed on them; small objects can also be moved onto or off these items. The subset covers 28 scenes and contains 226 subtasks, including 112 relocated-target subtasks. Among these, 66 589˙58\.9% involve movable supporting furniture. We evaluated IVAM-Nav and 3D-Mem on this subset.

**

Tablej5Qu−1Table j5Qu-1

Performance on extension subset with changing supporting surfaces.** The final column reports SR on relocated-target subtasks involving the added movable supporting furniture.

MethodSR ↑SPL ↑Relocated-target SR on changing supporting surfaces ↑
3D-Mem19.0%14.4%16.7%
IVAM-Nav (Ours)32.3%16.7%33.3%

As shown in Table j5Qu-1, IVAM-Nav achieves higher overall SR and SPL than 3D-Mem, and also performs better on relocated-target subtasks involving movable supporting furniture. Compared with the results on the original benchmark, overall SR decreases from 21.9% to 19.0% for 3D-Mem and from 33.4% to 32.3% for IVAM-Nav, indicating that the extension presents a more challenging setting. We have added the construction details and evaluation results of this extension in the Appendix. We will release the complete LiSoNav-Eval dataset, including both the original benchmark and the extension.

[Weakness 1 / Question 1] Navigation with Target-Only Instructions

Thank you for this suggestion. We have extended the target-only evaluation from the 245-subtask method-analysis subset to all 1,199 subtasks in LiSoNav-Eval, with separate results for the 545 relocated-target subtasks. In this setting, each instruction describes only the target object, without mentioning its supporting surface or nearby surroundings. For example, "Find the tissue box on the speckled countertop near the sink" becomes "Find the tissue box."

**

Tablej5Qu−2Table j5Qu-2

Full-benchmark evaluation with and without contextual cues.** Values in parentheses indicate percentage-point changes relative to the target + context setting.

MethodInstructionSR ↑SPL ↑Relocated-target SR ↑
3D-Memtarget + context21.9%15.5%17.2%
3D-Memtarget-only17.3% −46˙pp−4\.6 pp12.1% −34˙pp−3\.4 pp12.8% −44˙pp−4\.4 pp
IVAM-Nav (Ours)target + context33.4%18.4%30.5%
IVAM-Nav (Ours)target-only29.1% −43˙pp−4\.3 pp15.9% −25˙pp−2\.5 pp26.4% −41˙pp−4\.1 pp

As shown in Table j5Qu-2, target-only instructions reduce both SR and SPL for both methods compared with instructions that include surface and surroundings cues. This shows that contextual cues make the originally challenging navigation task easier. On relocated-target subtasks, IVAM-Nav achieves 26.4% SR with target-only instructions, compared with 12.8% for 3D-Mem. We have included these results in the Appendix.

[Weakness 2 / Question 2] Runtime, MLLM Calls, and Navigation Distance

Thank you for these helpful questions. We compare IVAM-Nav with 3D-Mem thestrongestnon−oraclebaselinethe strongest non-oracle baseline on the full LiSoNav-Eval benchmark. Both methods use the same hardware: an NVIDIA L20 GPU 45GiB45 GiB and Intel Xeon Platinum 8365A CPUs. All MLLM requests use the Qwen3-VL-Plus API through Alibaba Cloud. API waiting time measures the time spent waiting for MLLM API responses, while total runtime measures the time required to complete a subtask, including API waiting, local computation and planning, and simulator navigation.

**

Tablej5Qu−3Table j5Qu-3

Average computational cost and performance per subtask on the full benchmark.**

MethodOutput tokensMLLM callsTotal API waiting time ssTotal runtime ssSR ↑SPL ↑Navigation distance mm
3D-Mem3.04k22.2271.41180.2121.9%15.5%19.76
IVAM-Nav (Ours)1.45k28.0630.3095.9233.4%18.4%31.86

Output tokens, MLLM calls, and runtime. IVAM-Nav generates fewer output tokens per subtask than 3D-Mem 1\.45k vs\. 3\.04k, as it does not require lengthy chain-of-thought rationales. It makes more MLLM calls 28\.06 vs\. 22\.22 because it separately requests per-view descriptions and target verification. Since IVAM-Nav selects multiple complementary views at once, these independent requests can be submitted in parallel. Fewer output tokens reduce response-generation time and thus API waiting time, while parallel execution further reduces waiting time by overlapping independent requests. Together, these factors result in 57.6% less API waiting time and 46.8% shorter total runtime than 3D-Mem.

Navigation distance and Success weighted by Path Length SPLSPL. IVAM-Nav travels farther than 3D-Mem 31\.86 m vs\. 19\.76 m because it verifies whether the target is still at its remembered location and continues searching when it is absent. In contrast, 3D-Mem directly navigates to the retrieved location without checking whether the target is present, resulting in shorter paths but lower success rates. Despite its longer travel distance, IVAM-Nav achieves higher SPL 184˙18\.4% vs\. 15\.5%, which jointly accounts for navigation success and path efficiency.

API waiting time across the core MLLM stages. IVAM-Nav first revisits historical viewpoints for memory revalidation and target verification. If the target is not confirmed, it switches to active exploration, where the MLLM selects frontiers or supporting surfaces for further inspection. Table j5Qu-4 reports the average API waiting time for each core stage. Other API operations include auxiliary steps such as target-description normalization and rephrasing.

**

Tablej5Qu−4Table j5Qu-4

Average API waiting time per subtask across the core MLLM stages of IVAM-Nav.**

Core MLLM stageAverage API waiting time ss
Historical-viewpoint revalidation and target verification7.94
Exploration decision13.15
Multi-view surface description and target verification7.78
Other API operations eg˙,˙target−descriptionnormalizationandrephrasinge\.g\., target-description normalization and rephrasing1.43

Table j5Qu-5 reports the additional overhead introduced by multi-view inspection and memory revalidation. For multi-view inspection, we count the API waiting time and navigation distance for views acquired after the first view. For memory revalidation, we count consistency checks, local memory updates when inconsistencies are detected, and the associated re-observation cost.

**

Tablej5Qu−5Table j5Qu-5

Additional overhead introduced by multi-view inspection and memory revalidation per subtask.** Percentages are calculated relative to the average total API waiting time 303˙0s30\.30 s and navigation distance 318˙6m31\.86 m, respectively.

ComponentAdditional API waiting time ss% of total API waiting timeAdditional navigation distance mm% of total navigation distance
Multi-view inspection1.514.98%4.2913.47%
Memory revalidation0.652.15%0.2270.71%

We have included these computational-cost analyses in the Appendix.

[Question 3] Sensitivity to Viewpoint Revisitation Errors

Thank you for this constructive suggestion. To evaluate the sensitivity of IVAM-Nav to errors in revisiting historical viewpoints, we conduct experiments on the 245-subtask analysis subset for efficient evaluation. This evaluation is conducted without target reference images. At each revisit, we randomly sample horizontal position and heading offsets uniformly within the specified bounds relative to the stored pose. The agent then acquires a new observation from the perturbed pose.

**

Tablej5Qu−6Table j5Qu-6

Sensitivity to position and heading errors during historical-viewpoint revisits.**

Position / heading perturbation boundsSR ↑SPL ↑Relocated-target SR ↑Revalidation accuracy ↑
No perturbation31.4%18.1%30.8%77.5%
5 cm / ±5°30.6%17.5%29.9%76.3%
10 cm / ±5°31.4%16.8%31.6%74.7%
10 cm / ±10°31.0%17.1%29.1%73.9%
20 cm / ±10°30.2%16.6%29.1%73.2%

As shown in Table j5Qu-6, revalidation accuracy decreases from 77.5% to 73.2% as perturbations increase, since position and heading errors make the current observation harder to match with the stored view. However, this decrease has a limited impact on navigation success, with SR dropping only from 31.4% to 30.2% under the largest perturbation. This is because even when a revisit fails to confirm the target, IVAM-Nav can check other historical viewpoints or continue exploring to locate it. This mechanism helps mitigate the impact of revisitation errors on final navigation performance.

Thank you for the detailed response and the additional experiments. I appreciate the authors' efforts to address my concerns, particularly through the target-only evaluation, computational cost analysis, and localization error sensitivity experiments. These results provide stronger support for the proposed method and clarify several points raised in my review.

Overall, I believe my main concerns have been adequately addressed. I will maintain my original rating of 3 (Weak Acceptance)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.