- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work use a world model to predicts the most dangerous future scene, then uses a VLM to reason about safety guidance, and feeds both the predicted image and guidance into a VLA to help the robot avoid obstacles while completing the task.
Critical strengths
-
The paper is well motivated and targets at a critical problem: improving safety in VLA-based robotic manipulation.
-
The results show improvements in task success and collision avoidance on SafeLIBERO.
-
The paper includes real-robot demonstrations that provide initial evidence of practical applicability.
Critical weaknesses
Despite some promising results, the experimental design and comparisons have several flaws that limit support for the paper’s main conclusions. The evaluation is limited in benchmark coverage, and the ablation studies do not adequately justify the design choices.
-
The effect of the training data is not isolated. I could not find a pi0.5 baseline directly fine-tuned on the same SafeLIBERO demonstrations as ASR-VLA. This is missing of experiment is fatal. Comparing against a pi05 policy trained on regular LIBERO is not a fair comparison. Consequently, it remains unclear whether the improvements come from the proposed design or simply from learning obstacle-avoidance behaviors present in the training data.
-
Limited baseline comparisons. The paper evaluates only a narrow set of baselines and does not clearly show whether they receive the same SafeLIBERO training data as ASR-VLA, making it difficult to conclude the source of the gain. Moreover, neither safety-aware baseline is world-model-based. The absence of a comparison with closely related world-model-based methods, such as SafeDojo, leaves the contribution relative to existing approaches insufficiently established.
-
Limited evaluation scope. The evaluation relies on a single simulation benchmark with a limited set of tasks focused on tabletop obstacle avoidance. This narrow coverage does not sufficiently support the paper’s broader claims about safety-aware robotic manipulation. As is illustrated in many other research work, the conclusion can be very different between mobile and table-top settings
-
Limited real-robot evaluation. The real-robot evaluation covers only a few tasks and reports neither trial counts nor success and collision rates, making it difficult to assess whether the demonstrated benefits are statistically meaningful. Moreover, the experiment scenes looks uncluttered and requires minimal collision awareness.
Questions for authors
-
Can the authors report results for pi05 and OpenVLA directly fine-tuned on the same SafeLIBERO demonstrations as ASR-VLA, without future prediction or safety reasoning, to isolate the effect of the training data?
-
Can the authors evaluate the framework with a broader range of VLA backbones, such as X-VLA[1] and UniVLA[2], to assess whether its benefits generalize beyond pi05 and open-vla?
-
Can the authors include world-model-based safe-control baselines, such as SafeDojo, to establish the contribution relative to existing predictive safety approaches?
-
Both ablated variants achieve strong and similar performance. Can the authors explain why? This seems contradictory to claimed complementary roles of future imagery and safety reasoning. Does the incremental improvement from combining them justify the additional model training and inference costs?
-
Can the authors provide out-of-distribution evaluation by withholding some task types or an entire suite during training and evaluating on the held-out tasks? The current evaluation does not clearly establish whether the method learns transferable safety behaviors or overfits to the training task distribution.
[1] Jinliang Zheng et al., X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
[2] Yuqi Wang et al., Unified Vision-Language-Action Model