acceptodds
Withdrawn from ICLR 2027

LIBERO-Occ: Evaluating and Improving Vision-Language-Action Models under Scene-Induced Occlusion via Viewpoint Imagination

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models achieve strong performance on standard manipulation benchmarks, but most evaluations assume that task-relevant objects are fully visible. This assumption often fails in realistic settings, where occlusion makes manipulation partially observable. In this paper, we study scene-induced occlusion as a fundamental challenge for VLA models and introduce LIBERO-Occ, an occlusion-oriented extension of LIBERO. Experiments show that state-of-the-art VLAs suffer substantial performance degradation under occlusion. To address this issue, we propose Viewpoint Imagination (VIM), which generates a complementary view from an occluded primary observation and conditions action prediction on both observed and imagined evidence. VIM improves robustness across task suites, occlusion types, and severity levels without requiring additional cameras at deployment time, suggesting that viewpoint imagination is an effective mechanism for perception completion in partially observable manipulation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language-Action, Robotic Manipulation, World Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oNZBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new benchmark and a VLA model. The authors address the occlusion problem in the robot manipulation task. Given the LIBERO benchmark, the authors newly curate the data to have occlusion cases. The name of this curated dataset is LIBERO-occ.

As a next step, the authors newly train a VLA model. Especially, to deal with the occlusion problem at the exocentric viewpoint, the model "imagines" the top-view image. Technically speaking, the imagination is not something new and the authors clearly said so. Nonetheless, introducing the top-view imagination makes sense to me.

Critical strengths

S-1 Good motivation

It is quite important to deal with occlusion cases for the robot manipulation task. I pretty much like the motivation of this work. Moreover, I do agree that existing benchmarks, especially LIBERO, LIBERO-pro, LIBERO-plus, etc, did not deal with the occlusion cases. However, in the real-world demo, this is quite serious issues.

S-2 Good dataset curation

The curated dataset is simple, but it is good enough to demonstrate the results of existing VLA models that do not consider this occlusion problem during training.

Critical weaknesses

W-1 Unfair comparison?

It is a trick issue. Usually in LIBERO-plus and LIBERO-pro that also extends the LIBERO benchmark with the specific purposes, lots of recent VLA models trained with the original LIBERO and evaluate the models with the LIBERO-plus and LIBERO-pro benchmarks. So, it can be understood as "zero-shot" evaluation. This is a common practice in this field. Here, the problem comes. In the table 2, I wonder whether the authors also newly train or newly finetune the related studies (UniVLA, OpenVLA, ... PI0.5) with LIBERO-occ that the authors newly curated. I deeply check the paper, but the authors did not clearly provide the information about this. It is highly important from my side.

If the proposed method VIM is the only method finetuned with LIBERO-occ, it is totally unfair comparisons.

Moreover, following the conventions in this field, I wonder the performances of VIM that is purely trained with LIBERO benchmark. I truly understand that the LIBERO benchmark does not provide top-view images, but it is quite difficult to judge whether the imagination learning from LIBERO-occ is truly effective or not. At this moment, I cannot verify this issue through the paper.

W-2 Lack of information about generalization test.

The section 4.2.4 is about the generalization test. The authors claim that they evaluate the proposed model in different benchmark having occlusion (I guess). But it is hard to follow what it means. From Line 345 to Line 356, I cannot understand any single sentences at all. What existing datasets did you use for the generalization test? LIBERO-plus? Then, how did you sample evaluation episodes having occlusion cases? I also read the supplementary material, but this part is totally out of my understanding.

Since the authors propose a new method so the authors should evaluate the proposed method at least more than 2 benchmarks in my opinion. In that perspective, this highly confusing and hidden descriptions about the generalization test are not qualified as a ICLR submission.

I hope authors to revise this section in manuscript and the section in the supp entirely. This is quite an important issue as a reviewer.

Questions for authors

Overall, I enjoyed reading this paper. I also previously had a same idea and motivation of this task and the authors exactly address the problem through this submission. I have no doubts on the novelty of this paper. Honestly, the proposed method is highly naive but both the proposed dataset and the proposed method are well aligned with the motivation. I pretty much like this consistency.

Please refer to the weakness section and I hope to see the rebuttal soon.

1 comment
Reviewer_oNZBOpenReview ↗

No rebuttal?

Do the authors plan to upload the rebuttal?

Reviewer_Xb2DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LIBERO-Occ, which adds physical occlusions into LIBERO to obtain 2000 instances. Each instance is annotated by occlusion target and severity. The paper also proposes VIM, which builds on UniVLA to generate a complementary view from the main view before predicting actions. With only the main view, VIM reaches 65.1% on LIBERO-Occ versus 57.1% for the backbone-matched UniVLA.

Critical strengths
  1. A novel designed physically grounded benchmark: Occluders are placed in the 3D scene and filtered by visibility, collision, and replay checks. Figure 3 shows that the real wrist view helps by only 2–8 points on LIBERO but by 22–46 points on LIBERO-Occ, confirming that the benchmark removes task-relevant information. Since existing LIBERO variants do not treat visibility as a controlled factor, this is the main reason I see value in the paper.
  2. An interesting transfer scenario: Feeding VIM-generated wrist views to OpenVLA-OFT and UniVLA improves them. This is the most direct evidence in the paper that the imagined views themselves carry useful information.
Critical weaknesses

W1: The Gain Is Not Attributable to Imagination

  • Stage 1 trains view generation on 3,539 replays that include occluded scenes, while all baselines, including the backbone-matched UniVLA, are trained only on original LIBERO and never see an occluder.
  • Without Stage 1, VIM drops to 36.3% in Table 6, 19 points below UniVLA under the same 100-rollout protocol (55.3%, Table 5). VIM's advantage therefore depends almost entirely on Stage 1, which adds both more view data and occluded scenes, and the two effects cannot be separated.
  • Two other explanations are not ruled out: the view loss may only act as an auxiliary representation objective without the generated view being used at inference, or the model may memorize LIBERO's nearly fixed layouts, where the location of a hidden object can be guessed from the task.
  • Since the methodological contribution rests on attributing the gain to imagination, this is the main reason for my negative recommendation.

W2: The Camera-Free Motivation Does Not Hold

  • The imagined view is the wrist camera, yet training VIM requires real wrist images, so the robot used for data collection must have a wrist camera anyway. LIBERO also provides one, which the authors remove from the input and then ask the model to imagine back.
  • When the wrist camera is available, using it directly works better. Adding the gaps in Figure 3 to Table 2, the real wrist view gives about 79% for UniVLA, 81% for π-0, and 86% for π-0.5 on LIBERO-Occ, versus 74% for VIM. Table 2 calls the VIM row with the real view a reference upper bound, so even perfect imagination leaves VIM below baselines that use the camera directly.
  • VIM is thus useful only when no wrist camera is present, while its training requires one, which limits its practical value.
Questions for authors

Questions 1 to 3 could change my rating; Question 4 concerns the validity of the benchmark.

  • Q1: Regarding W1, how does UniVLA perform on LIBERO-Occ when trained with the same occluded scenes as Stage 1, e.g., the successful occluded replays that carry action labels?
  • Q2: Regarding W1, how does VIM perform when trained with the view loss but evaluated without generating the view? How accurately is the occluded target placed in the generated views?
  • Q3: Regarding W2, what are the baselines' LIBERO-Occ success rates when the real wrist view is available?
  • Q4: What is the success rate when the same occluder is placed at a position that does not occlude the target? This would separate the effect of occlusion from that of an additional object in the scene.
Reviewer_rcz2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces LIBERO-Occ, a LIBERO extension for evaluating VLA models under scene-induced occlusion. It places occluding objects in 3D scenes and filters the scenes to ensure that the original tasks remain executable. The paper also proposes VIM, which generates a wrist view from the main view and uses both views for action prediction. The main results show higher success on LIBERO-Occ than the tested single-view baselines. The authors also evaluate on original LIBERO, an OOD occlusion split, and LIBERO-Plus.

Critical strengths
  1. LIBERO-Occ is a useful benchmark contribution. The occluders are placed in the scene rather than added as image masks. The authors check that the target is occluded, the new object does not collide with other objects, and the original demonstration still succeeds. This makes the benchmark a focused test of occlusion while keeping the tasks executable. I see this as the strongest part of the paper.
Critical weaknesses
  1. The practical motivation for VIM and the evidence for its benefit are not fully convincing. The main setting assumes that the wrist view is unavailable at inference, although LIBERO provides this view and VIM requires paired wrist images during training. With a real wrist view, VIM improves from 65.1% to 74.0% on LIBERO-Occ. The paper does not make clear how important the wrist-unavailable setting is in practice. VIM also receives additional occluded, paired-view episodes in Stage 1, while the baselines do not receive the same visual training data. The comparison therefore does not clearly separate the effect of viewpoint imagination from the effect of additional occlusion exposure and supervision.
  2. The severity results are difficult to interpret as a test of increasing difficulty. In Figure 4, Heavy occlusion gives higher success than Medium occlusion for VIM and UniVLA on Goal, Spatial, and LIBERO-10. The severity measure is based on the fraction of target area hidden from the main camera, but task difficulty also depends on which target is hidden and on the task itself. For example, Table 1 shows that dual occlusions are much more common in Medium than in Heavy. LIBERO-Occ is still useful as an occlusion benchmark, but the current analysis does not support a clear conclusion about robustness as occlusion difficulty increases.
Questions for authors
  1. Clarification that could affect my assessment: What are the intended deployment cases where paired wrist images are available for training but a wrist camera cannot be used at inference? How should VIM’s benefit be weighed against using the real wrist view in these cases?
  2. Additional evidence that could affect my assessment: Can the authors provide a primary-view control trained with the same Stage-1 occluded episodes and comparable compute, but without using an imagined wrist view for action prediction? This would help identify the source of VIM’s gain.
  3. Reanalysis that could affect my assessment: Can the authors report the Figure 4 results by task template and occlusion target type, with sample counts and uncertainty? Does the relation between occlusion ratio and success remain after accounting for these factors?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.