acceptodds
Withdrawn from ICLR 2027

It Is Not Seeing the Hazard: A Frozen Vision-Language Safety Score Measures Its Caption Bank

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Frozen vision–language models increasingly provide safety signals for reinforce- ment learning. Their use assumes that similarity to language describing danger indicates the hazard itself. Yet policy return and collision rate cannot reveal whether a score detects hazards or responds to correlated features of the scene. VLM-based methods have reported gains in driving and safe-RL benchmarks by converting image–text similarity into rewards, costs, or confidence weights. Such signals promise to reduce reliance on manually designed feedback. They may also reflect prompt structure, embedding geometry, or camera viewpoint, leav- ing their safety meaning unverified. To address this gap, we present a controlled evaluation of a frozen CLIP prompt-margin safety score. We apply the score to trajectories generated by policies that never receive it, match pre-contact observa- tions to contact-free observations with comparable hazard geometry, and vary the captions, encoder, and camera view. Across three policies, 180 episodes, and 130 isolated contact onsets, the score decreases for about twenty steps before contact. Mechanism controls indicate that the score mainly tracks resemblance to the scene shared by its captions and changes with caption separation and camera view. A constant-confidence control retains the lower catastrophe-rate point estimate, so policy gains do not establish hazard perception.

Primary area:
interpretability and explainable AI
Keywords:
safe reinforcement learning, vision-language models, CLIP, constrained MDP, cost shaping

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9BMSOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigate whether CLIP can detect hazards in simulated environments and find that its safety score mainly reflects scene similarity, which is not particularly surprising given CLIPs known limitations.

Critical strengths

I do not consider this paper has any strengths to compensate for the limited novelty, and methodological concerns discussed bellow.

Critical weaknesses

My main concern is that most findings are already well known. It is not surprising that CLIP is sensitive to prompts, has problems with negation etc. These limitations have been studied before, and the authors even cite several papers. Repeating known limitations in a safe-RL environment is not, by itself, a sufficient contribution for ICLR.

Besides, the Results section is extremely complicated and hard to read and follow which makes the claimed contribution impossible to evaluate. There are many different configurations, statistical comparisons, and ablations, but the actual message is just unclear. I bet an LLM reviewer would like it, but for a human it is, in this form, really difficult to understand which observations are fundamental and which are specific to the experimental setup.

Following on that, I also have some serious doubts about the methodology. The main results rely on an old CLIP model (released in 2021) evaluated on synthetic data. CLIP was not specifically trained on such data, and its limitations under domain shift are already well studied. Therefore, I am not sure why the authors find its behavior so surprising. Without properly investigating the effect of domain shift, how can we know whether the observed results are caused by the proposed scene-template mechanism or simply by using a model that is not well suited for this domain? The limited experiments with other models do not really answer this question.

Last but not least, I find it quite disrespectful to the whole community that, while we are on one hand discouraged from using LLMs to assist with reviews for example, the authors submit a manuscript that reads like it was written almost entirely by Claude. The authors acknowledge AI-assisted drafting, but it is not clear how far this assistance actually went. Was AI also used to implement, run, and analyze the experiments? If so, were the results independently verified? Given how difficult the Results section is to follow, I find this particularly concerning. I have nothing against using LLMs as assistants, but ... reviewers should not have to spend hours trying to understand and verify AI slop.

Questions for authors
  • Q1. What is the main novel contribution beyond already known CLIP limitations?
  • Q2. Why do the main experiments rely on CLIP ViT-B/32 from 2021?
  • Q3. How do you distinguish the proposed scene-template mechanism from domain shift caused by synthetic data?
  • Q4. Did you verify that CLIP can reliably recognize relevant objects and hazards in these environments?
  • Q5. Why are the main mechanism experiments not repeated with modern generative VLMs?
  • Q6. How representative are the results for current VLM-based safety methods?
  • Q7. Why do the results differ so much across camera views and environments?
  • Q8. Why was the closed-loop evaluation not repeated on genuinely held-out scenarios?
  • Q9. What conclusions can be drawn from the constant-confidence experiment when the difference from the deployed model is statistically unresolved?
  • Q10. What is the practical significance of these findings for developing or evaluating modern VLM-based safety systems?
Reviewer_vNy7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an empirical audit of frozen vision-language model (VLM) safety scores used in safe reinforcement learning. Rather than proposing a new learning algorithm, the authors investigate whether CLIP-based safety scores genuinely detect approaching hazards or primarily respond to scene characteristics associated with their caption banks. The study separates score evaluation from policy behavior through passive-observer experiments and examines the effects of hazard geometry, caption semantics, embedding structure, and camera viewpoint. It further introduces constant-confidence and temporally mismatched controls in closed-loop reinforcement learning. The findings suggest that the tested CLIP safety score is strongly influenced by scene-template similarity and observation viewpoint, while improvements in downstream policy safety metrics do not necessarily establish genuine hazard perception.

Critical strengths
  1. Important research question and careful experimental isolation. The paper examines a fundamental assumption underlying the use of frozen VLMs as safety signals. Separating the safety score from the policies generating the trajectories is a valuable experimental design, as it reduces the possibility that the observed pre-contact patterns arise from feedback between the score and policy behavior.
  2. Comprehensive mechanism-oriented analysis. The authors provide multiple complementary controls involving hazard geometry, caption modifications, alternative encoders, camera viewpoints, and closed-loop confidence replacements. The analysis also discusses negative findings, statistical limitations, and reproducibility issues. These experiments provide useful insights into the limitations of interpreting image-text similarity as semantic hazard awareness.
Critical weaknesses
  1. The proposed scene-template explanation is not fully established through causal isolation. Although the results are consistent with the scene-template hypothesis, the evidence does not fully isolate the visual factors responsible for the pre-contact score changes. The geometric matching experiments primarily control hazard distance, bearing, and time-to-contact, but do not sufficiently control vehicle alignment, heading, track geometry, or other scene factors that may jointly influence collision risk and image-text similarity. Furthermore, although the contrast-free score based on the common caption direction reproduces a substantial portion of the observed effect, this association does not directly establish the causal mechanism of the original prompt-margin score. A more controlled intervention that independently varies track alignment or scene appearance while preserving hazard geometry would strengthen the central explanation.
  2. The closed-loop evidence and cross-model generalization remain limited. The constant-confidence experiment provides an important control, but the absence of a statistically significant difference between constant and frame-dependent confidence does not establish equivalence. The experiments involve a limited number of training seeds and only one closed-loop environment and algorithm. Additionally, the reported evaluation-scenario aliasing limits the interpretation of the absolute performance results as estimates of held-out generalization. The strongest mechanism evidence concerns a particular CLIP configuration and caption bank. Since changing the encoder or caption set can substantially alter the danger-score behavior, it remains uncertain to what extent the proposed interpretation generalizes across different VLM-based safety scores.
Questions for authors

Q1. Can the authors further control for vehicle lateral displacement, heading error, speed, and track geometry when comparing pre-contact and contact-free observations? Alternatively, could the scene-template hypothesis be evaluated using controlled rendering interventions that preserve hazard geometry? Q2. Given that the prompt margin mathematically cancels the exact common caption direction, how should the finding that a common-direction score reproduces 73% of the effect be interpreted mechanistically? What additional evidence distinguishes a scene-template mechanism from correlated but distinct embedding responses? Q3. Can the authors provide additional statistical evidence regarding the comparison between constant and frame-dependent confidence, including confidence intervals for their difference and the detectable effect size? Would the findings persist under evaluation on genuinely held-out scenarios? Q4. Does the scene-template explanation remain valid for CLIP ViT-L/14, where the danger-group score increases before contact? How much of the mechanism analysis transfers across encoders and caption sets? Q5. Can the authors clarify how the contributions of this work extend previous findings that improvements in safe reinforcement learning do not necessarily indicate collision anticipation?

Reviewer_vS9sOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper looks at a specific VLM-RL setup where a frozen CLIP ViT-B/32 scores each driving frame against four “safe” and four “danger” captions, and the margin between the two groups is used as a safety signal. The authors check what this score actually responds to. They run it on trajectories from policies that never used it and find it drops about 20 steps before a collision. Then they change the captions, the camera, and the encoder, and argue that the drop comes from the frame no longer looking like “a racecar on a track,” not from the model seeing the hazard. They also train policies with the score replaced by a constant and get the same catastrophe rate. Along the way they report four bugs in the Safety-Gymnasium / MetaDrive evaluation stack.

Critical strengths

The four defects found in the Safety-Gymnasium / MetaDrive evaluation stack (scenario aliasing, traffic re-randomization, hardware-dependent replay, shaping bonus in logged return) are useful to anyone running these benchmarks and would make a good standalone technical note.

Critical weaknesses
  • The audited reward signal is not representative of current practice. Cosine similarity between a frozen CLIP ViT-B/32 embedding and eight hand-written captions is a weak reward model by today’s standards; the field has largely moved to querying large VLMs as reward or preference judges, and the relevant requirement for RL is reward accuracy rather than per-step cost. Showing that this particular construction does not perceive hazards therefore says little about the reward models people actually use. The one 7B generative model in the paper is used only to show that its margin also falls, not to ask whether it sees the hazard. The title’s claim about “a vision-language safety score” seems to be broader than the evidence.
  • The result is predictable before the first experiment. That CLIP similarity to “the racecar is about to crash” tracks scene content rather than danger follows from the bag-of-words and negation papers the authors themselves cite, and that a constant reward bonus trains the same policy as a slowly varying one is reward shaping. The experiments confirm this carefully, but confirmation of an expected null is a short note rather than a conference paper, and the paper’s own numbers do not go further than that: pixel variance recovers half the effect and the margin does not significantly beat it (Appendix E.6).
Questions for authors
  1. Could you clarify the intended audience for this audit? If there are current systems that still use the CLIP prompt-margin construction as a safety reward, pointing to them would help establish why the result matters today.
  2. On the same pre-contact frames, does a current large VLM queried as a judge (e.g., “Is the car about to collide?”) detect the hazard? If it does, that would be the result the title promises.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.