acceptodds
Withdrawn from ICLR 2027

From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied agents exploring indoor environments require reliable semantic occupancy memory that persists across observations and revisits. Building such memory is challenging because each observation provides incomplete and uncertain geometric and semantic evidence. We introduce GEM-Occ, a Gaussian Evidence Memory framework that consolidates evidence accumulated over time into persistent semantic occupancy memory. Local predictions are converted into occupied semantic Gaussians and free-space ray evidence. Confidence- and visibility-aware causal updates integrate supporting observations, suppress occupancy contradicted by observed free space, and preserve previously observed structures through occlusion. A hierarchical memory organization supports continued mapping and efficient queries across connected indoor spaces. To evaluate this capability, we introduce HIOcc, a unified benchmark for embodied semantic occupancy memory. HIOcc establishes a shared semantic label space and evaluation framework spanning local prediction, room-level online mapping, and building-level mapping, while accommodating perspective and panoramic observations. Experiments on HIOcc demonstrate that GEM-Occ outperforms existing methods, enabling accurate semantic occupancy prediction and consistent online mapping across spatial scales with efficient memory usage and fast occupancy queries.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Semantic Occupancy, Streaming Map

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_A79VOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GEM-Occ, a framework for persistent semantic occupancy mapping that integrates Gaussian-based geometric and semantic evidence through incremental memory updates. It also introduces HIOcc, a unified benchmark for local occupancy prediction, room-level mapping, and building-level exploration.

Critical strengths

The proposed framework is reasonably designed, and addressing a meaningful problem of maintaining and updating a persistent occupancy representation during continuous exploration.

Critical weaknesses
  1. Limited methodological novelty. GEM-Occ largely builds upon existing work, including VGGT and Gaussian-based occupancy methods, while introducing hand-crafted implementations for memory fusion and updates. The overall framework appears to be primarily a system-level integration rather than a novel methodological contribution.
  2. Unclear significance of the proposed HIOcc benchmark. Building on the previous concern, the proposed reference solution relies heavily on engineering-oriented fusion and memory management strategies. This raises doubt that whether the benchmark motivates new learning approaches, or primarily encourages such engineering implementations.
  3. Modest result improvements. In Table 3, GEM-Occ improves occupancy IoU over GPOcc by only 0.68, and the improvement in mIoU is heavily dominated by the ceiling category while the improvements across most other categories are relatively small. The qualitative visualization in Figure 4 also show limited improvements.
  4. Missing relevant literature. The paper overlooks several relevant studies on Gaussian-based occupancy prediction and learning, including:
    • GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding (CVPR 2025).
    • GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flow (ICCV 2025).
    • GaussRender: Learning 3D Occupancy with Gaussian Rendering (ICCV 2025).
Questions for authors

See Weaknesses

Reviewer_YXmyOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes GEM-Occ for online indoor semantic occupancy mapping with known poses. Each observation is converted into semantic Gaussians (with ray-aligned, depth-uncertainty covariance) and free-space ray evidence. The paper also introduces HIOcc, a benchmark built from ScanNet, ScanNet++, and Matterport3D. GEM-Occ reports gains over GPOcc and EmbodiedOcc on all three tracks, along with lower memory and query latency than a flat Gaussian memory variant.

Critical strengths

The paper proposes a method for online semantic occupancy mapping.

The paper introduces a new dataset/benchmark, HIOcc.

Critical weaknesses

Limited novelty. Most components of GEM-Occ closely resemble classical probabilistic mapping.

Insufficient comparisons. All results are reported only on the self-built HIOcc benchmark. There are no results on Occ-ScanNet or EmbodiedOcc-ScanNet, which is more common used as in GPOcc or EmbodiedOcc.

The improvment over GPOcc differs greatly, ceiling (+20), while others around 1-2 points, the author should explain this.

Questions for authors
  1. Why don't conduct experiments on Occ-ScanNet or EmbodiedOcc-ScanNet?
  2. Why GEM-Occ performs much better on ceiling?
Reviewer_gHHROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GEM-Occ combines semantic Gaussians and free-space evidence into persistent occupancy memory, organized through submaps and maintenance rules. It also introduces HIOcc for local, room-level, and building-level evaluation. Results are promising, but benchmark definitions and ablation controls need clarification.

Critical strengths
  1. Distinguishing free space from occlusion is well motivated: the method can revise contradicted occupancy while retaining temporarily hidden structures.

  2. HIOcc brings perspective and panoramic observations into a shared taxonomy across three spatial scales. The annotation study is a useful addition.

  3. Evaluation includes accuracy, revisit consistency, storage, and runtime. The supplement also provides external building-level baselines and pose/depth perturbation tests.

Critical weaknesses
  1. Important settings remain unspecified, including fusion thresholds, covariance scales, training unroll length, and optimizer settings. These details matter for reproducing the method.

  2. Removing hierarchy drops single-frame mIoU from 57.76 to 52.41 (Table 6), so this ablation also changes local prediction quality. Frozen-predictor ablations would better isolate the memory components.

  3. View-dependent filtering may improve annotation consistency partly by removing difficult regions. Please report retained geometry coverage and clarify how per-view targets form global evaluation masks.

  4. Progress AUC and revisit consistency lack precise computation rules (Appendix C.4). Agreement can also reward repeated mistakes, so I'd like ground-truth revisit accuracy alongside consistency.

  5. Matched confidences are summed, while free-space correction is applied once per observation (Appendix B.3). Sampling-density and repeated-view tests would clarify whether this imbalance makes incorrect occupancy difficult to reverse.

  6. The reported 5.2 FPS and 33.8 ms query latency lack hardware and query-size specifications. Memory accounting should also clarify whether the free-space cache and indexing overhead are included.

Questions for authors

See Weaknesses.

Reviewer_JddFOpenReview ↗
Rating
1
Confidence
2
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes: 1) GEM-Occ, an online semantic occupancy method with a VGGT-based encoder that converts each frame into per-pixel depth, class, and confidence; surface points become ray-aligned semantic Gaussians that are matched to existing memory and merged with confidence weighting. And 2) HiOcc, a dataset that combines re-rendered ScanNet with Occ-ScanNet's 11+1 label space, adds ScanNet++ and Matterport3D panoramic track. The paper claims better accuracy, greater revisit consistency, lower memory usage, and lower query latency than a flat Gaussian memory.

Critical strengths
  1. Clear hierarchy of the framework with a detailed description of each component. The probabilistic formulations look correct.
  2. Two parts of HIOcc are new and could be useful: the building-level panoramic track on Matterport3D (multi-room sequences with revisits) and occupancy targets on ScanNet++ geometry.
  3. Metrics that look at the map over time seem like an interesting idea (ProgressAUC and Revisit Consistency).
Critical weaknesses
  1. Reproducibility is very questionable. One of the main contributions is the dataset; another is the framework. No splits are given; no values for hyperparameters are provided; no hardware setup, training time, or GPU info. The paper doesn't state whether the baselines were retrained, as the number differs from that in the original papers.
  2. The following concern: is it practical on a robot? The title and abstract claim it is about embodiment, but nothing "embodied" appears in the paper. The paper is about offline mapping from posed videos. A.3 appendinx shows a drop in metrics under translation and perturbations, which contradicts the "robustness" statement.
  3. The benchmark's novelty is overclaimed. The ScanNet part is taken from Occ-ScanNeT and EmbodiedScan, which have labels on Matterport and ScanNet++, but are omitted from the table.
  4. ProgressAUC and Revisit Consistency are defined as benchmark metrics but aren't posted for other baselines where they are important. The same applies to memory metrics; they aren't defined and are applied only to the authors' options.

The writing is repetitive and weird. The introduction repeats the abstract; the phrase "reinforce supported structures, suppress contradicted occupancy, retain occluded memory" appears several times in the paper.

Questions for authors

Will splits, hyperparameters, label maps, and baseline configs be released? Can some experiments be run in simulators like Habitat? Can all the claims be aligned with what the paper presents, and are important benchmark metrics presented for all the baselines?

The method looks interesting, but the dataset paper must include all the details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.