acceptodds
Withdrawn from ICLR 2027

Learning Memory-Guided Scene Graph Reasoning for Temporal 3D Semantic Scene Completion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

3D Semantic Scene Completion (SSC) aims to infer scene geometry and semantics from partial observations, providing dense 3D scene understanding for autonomous driving. Existing completion methods do not consistently adapt their completion strategies to different observation conditions and make limited use of object-level occlusion relations to recover missing scene information. However, occlusion limits the available visual evidence, and simply aggregation multi-frame features does not explicitly model directed occlusion relations between objects. To address these challenges, we propose MemGraphSSC, a temporal SSC framework that combines observation-conditioned memory with occlusion-relation-guided refinement. We introduce two complementary components: (1) an Observation-Conditioned Dual Memory module that retrieves historical region features for history-supported voxels and learned class prototypes for observation-unsupported voxels, while preserving visual initialization for current-supported voxels; and (2) an Occlusion-Relation-Guided Refinement module that uses depth-consistent relations from precomputed scene graphs to construct voxel priors and approximate object supports, then refines features in occluded regions through directed messages between relation endpoints. Experimental results show that the proposed method achieves superior performance on SemanticKITTI and SSCBench-KITTI-360 datasets, outperforming existing state-of-the-art vision-based SSC methods.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Semantic Ccene Completion, Camera-Based 3D Scene Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_q5zoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemGraphSSC, a temporal 3D semantic scene completion framework that combines observation-conditioned dual memory with occlusion-relation-guided feature refinement. The method retrieves historical features for history-supported voxels and semantic prototypes for observation-unsupported voxels, while leveraging precomputed scene graphs to guide feature refinement through directed occlusion relations. Experiments on SemanticKITTI and SSCBench-KITTI-360 demonstrate competitive performance against existing methods.

Critical strengths
  1. The observation-conditioned memory mechanism addresses the varying availability of historical evidence across voxels, providing a reasonable strategy for integrating temporal information and semantic priors.
  2. The framework combines temporal memory with explicit occlusion relations, enabling more structured feature refinement in partially observed scenes.
  3. The method achieves state-of-the-art performance on two benchmarks, and the ablation studies demonstrate the effectiveness of the proposed components.
Critical weaknesses
  1. Although occlusion reasoning is a central contribution, the paper mainly reports overall IoU and mIoU. It lacks quantitative evidence showing improvements specifically in occluded regions.
  2. The experiments do not sufficiently demonstrate whether directed occlusion relations provide benefits beyond simpler alternatives, such as bounding-box priors or undirected feature aggregation.
  3. Scene-graph construction relies on additional large models, including Grounding DINO and Qwen3-VL-8B. However, the paper does not report end-to-end inference latency or graph-generation overhead, raising concerns about practical deployment.
Questions for authors
  1. Could the authors provide quantitative evaluations specifically on occluded regions to better demonstrate the effectiveness of the proposed occlusion-guided refinement?
  2. How does the proposed directed relation reasoning compare with simpler alternatives, such as undirected graph aggregation or bounding-box-based semantic priors?
  3. Could the authors report the computational overhead, including scene-graph generation time and end-to-end inference latency, compared with existing temporal SSC methods?

If the authors can address most of my concerns, I would be willing to raise my score.

Reviewer_a6XJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

MemGraphSSC builds on FoundationSSC for temporal semantic scene completion. It introduces Observation-Conditioned Dual Memory and Occlusion-Relation-Guided Feature Refinement (OFR) to combine historical evidence, semantic prototypes, and directed occlusion relations.

Critical strengths
  1. The method achieves 20.90% and 22.55% mIoU on SemanticKITTI and SSCBench-KITTI-360, respectively. Table 3 also reports a 2.22 point validation improvement over the FoundationSSC baseline.
  2. Table 4 shows that the proposed configuration outperforms both single source memory configurations. This provides experimental support for combining historical evidence and semantic prototypes.
Critical weaknesses
  1. Efficiency is important for temporal SSC, particularly in autonomous driving. The paper reports parameter increments but no latency or memory measurements for the complete pipeline, including Grounding DINO base, Qwen3-VL-8B-Instruct, and depth verification. Without these measurements, the practical cost of marginal gains from memory and OFR, namely 0.31 and 0.47 mIoU points in Table 3, cannot be assessed.
  2. The routing contribution is not fully isolated. Table 4 changes both the available memory sources and their assignment, making it difficult to distinguish the benefit of using both sources from that of observation-conditioned routing. A comparison retaining both sources while changing only their assignment is needed.
  3. The benefit of relation direction remains unclear. Although Table 3 includes an ORA ablation, it does not test whether the direction of occlusion relations accounts for improvement while preserving object information and supervision. Quantitative evaluation restricted to occluded regions is also missing.
  4. Figure 1 omits FoundationSSC, the direct baseline. Including its performance would make the improvement over the framework on which MemGraphSSC is built directly visible.
Questions for authors

Considering the limitations above, a weak rejection is proposed.

Reviewer_Dmj7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MemGraphSSC for camera-based temporal SSC. It (a) partitions voxels into current-supported / history-supported / observation-unsupported states and routes each to visual seed, episodic-memory retrieval, or learned class prototypes; and (b) injects object-level occlusion relations from a frozen VLM (Qwen3-VL-8B) as voxel priors and directed messages over occluded regions. SOTA-level results are reported on SemanticKITTI and SSCBench-KITTI-360.

Critical strengths

1.Using object-level occlusion relations instead of pure feature aggregation is a relatively interesting direction for SSC. 2.Presentation is clear with good figures.

Critical weaknesses

1.In Figure 1, "Input RBG Images" should be "RGB Images"; the authors should check the manuscript carefully.

2.Using FoundationStereo to obtain depth maps is a relatively old approach. Why not use a state-of-the-art depth estimation model?

3.SemanticKITTI and SSCBench-KITTI-360 are essentially datasets in the same KITTI-style format. Have the authors considered other datasets, such as nuScenes?

4.The paper lacks a comparison of inference time and efficiency.

5.The occlusion directions output by Qwen3-VL are merely the results of language/visual reasoning and are not equivalent to geometric ground truth. The authors are encouraged to compare their approach with alternative models and, if possible, benchmark the VLM-predicted occlusion directions against geometric ground truth to verify the validity of the proposed prior.

Questions for authors

1.In Figure 1, "Input RBG Images" should be "RGB Images"; the authors should check the manuscript carefully.

2.Using FoundationStereo to obtain depth maps is a relatively old approach. Why not use a state-of-the-art depth estimation model?

3.SemanticKITTI and SSCBench-KITTI-360 are essentially datasets in the same KITTI-style format. Have the authors considered other datasets, such as nuScenes?

4.The paper lacks a comparison of inference time and efficiency.

5.The occlusion directions output by Qwen3-VL are merely the results of language/visual reasoning and are not equivalent to geometric ground truth. The authors are encouraged to compare their approach with alternative models and, if possible, benchmark the VLM-predicted occlusion directions against geometric ground truth to verify the validity of the proposed prior.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.