- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies safety for agentic VLMs, which can call tools such as web search and image cropping. Authors argue that existing VLM safety benchmarks didnot test this, because the risk can usually be read off the image and request. To this end, they build SAVIT, a benchmark of 739 image-request samples in 8 safety categories. Each unsafe sample has a safe twin that differs in one detail. The correct response depends on a fact that has to be found with tools, such as a local law or a small label in the image. The samples are generated by two LLMs, reviewed by hand, paired with web or generated images, and filtered to remove those a small model can already answer without tools.
The main idea is to score two things separately: whether the final response states the safety-critical facts, and whether it responds appropriately.. The authors evaluate 15 VLMs with and without tools and find that joint success rate is low. Models state the key facts far less often than they respond appropriately, and tool help mainly by improving the first metric. The failure analysis says that most failures come from models not calling a tool, or calling it without retrieving the needed fact.
Critical strengths
The two-axis evaluation is the right way to look at this problem. A single ASR cannot tell a model that found the key fact from a model that knew nothing and refused by default. Separating "did the model get the fact" from "did it respond well given the fact" shows where failures come from.
The problem is timely and the evaluation is broad. There is no good benchmark for safety in agentic VLMs. The paper tests many recent models with and without tools, and checks that the conclusions hold under a different judge and a different screening model. This makes the main finding hard to dismiss: current models often fail to look for the evidence they need.
Critical weaknesses
The data generation pipeline is not good enough. Scenarios are written by LLMs and checked by hand, and the only automatic filter is one small model without tools. The paper gives no statistics on how many candidates were lost at each stage, how diverse the scenarios are, or which tools each sample needs. Many samples also depend on local rules or facts that change over time, so the labels are vague and a score may reflect whether a model found one specific fact. I cannot tell the data quality from the paper.
The tool design is thin and has little to do with VLMs. The benchmark has only a handful of tools, and only the zoom-in tool is vision-specific. Web search, page fetching and a code interpreter are the same tools an LLM agent would use. The setting is therefore close to an LLM tool-use setting with an image attached, and the paper does not show otherwise. The scope is also narrow for an "agentic" benchmark: single-turn, information-seeking tools only. A benchmark paper should offer a richer and better justified tool set. For VLM safety, we may need the intention hidden deeper in the text-image combination, with harder tools sets and longer reasoning chain.
The first metric is unfair to safe samples. A fact counts as acquired only if the final response states it. For unsafe samples this is reasonable to ask the model to figure out the underlying violated policy. For safe samples the model only has to comply, nothing asks it to check safety, and it may already know the rule. A correct, compliant answer is then scored as a failure. The authors admit the criterion is conservative, but they do not show how it still supports their conclusions.
Questions for authors
-
Which tools does each sample actually need, and how often is each tool used? Please report the split between samples that need image cropping, web search, or both, and the per-tool call statistics for the evaluated models. Were models restricted to the same four fixed tools for all instances?
-
Can you show that the benchmark tests something specific to VLMs? Is the zoom-in/image-cropping tool the only component that is specific to VLMs? If so, please explain how the proposed method and benchmark differ from prior work on agentic LLM safety.
-
How would the acquisition result change on safe samples if you separated "model knew the fact" from "model did not know"? For example, probe the model without the image for the relevant rule, and report appropriateness conditional on acquisition, so we can see how much of the 76% comes from the measurement rule.
-
How was the data pipeline filtered in practice? Please report the number of candidates produced, rejected in manual review, lost in image acquisition and removed by screening, and some measure of scenario diversity.
4 comments
Response to Reviewer QJeN (2)
W2. The tool design is thin and has little to do with VLMs. ...
Q2. Can you show that the benchmark tests something specific to VLMs? ...
SAVIT is vision-centric: in most scenarios, the safe and unsafe samples differ only in the image, so the task cannot be reduced to LLM tool use with an image attached. Its standard tool set is sufficient, and it already exposes the main failure of tool-using VLMs. We add these analyses to the paper.
1. Only zoom-in is vision-specific, close to LLM tool use → In SAVIT, web search and the code interpreter are also image tools. In LLM tool use, the text says what to look up; in SAVIT, requests refer to things like "this animal" or "the items shown," and what to search for often exists only in the image.
- Most web searches start from the image: 89% of 240 random queries follow a zoom-in or image-processing call or search for something found only in the image (e.g., an identified species, product, or place, or text read from it).
- The code interpreter processes the image: it runs Python on the input image (Appendix E), and 81% of its 3,659 calls operate on it.
- Zoom-in is the most used tool (40.9% of samples on average vs. 37.5% for web search), and models that zoom in more acquire more safety information (Spearman's ρ = 0.63; Appendix H.5).
- Often, only the image separates safe from unsafe: in 63% of scenarios (165 of 263), the safe and unsafe requests are word-for-word identical, so a text-only agent sees the same input and cannot tell them apart. In 55% of annotated samples, the decisive cue is text in the image that must be read closely (e.g., a label, form, or screenshot).
2. Small, under-justified tool set → The tool set is standard, sufficient, and not the bottleneck. It follows prior tool-using VLM work. With exactly these tools, a research agent found the annotated safety information for 95.1% of samples (Appendix H.4). Models fail because they barely use the tools (1.2 vs. 6.0 searches per sample), not because tools are missing; adding tools would not fix this and would mix safety with proficiency in specific tool APIs.
3. Single-turn, information-seeking scope → It targets the main failure of agentic VLMs. A single request still requires a multi-step agentic chain (inspect the image, identify the entity, search, read, and decide); the research agent used about 10 search and page-reading calls per sample. Information seeking is also the first step of any safe action: acquisition failures account for 94% of all joint failures (Figure 7(a)). SAVIT isolates this dominant failure.
4. Deeper hidden intent, harder tools, longer chains → SAVIT already hides the risk deeply, and frontier models fail it. Requests contain no harmful intent; the risk lies in external facts about what the image shows, which only tools reveal. Without tools, even the best model obtains this information for only 26.8% of samples; with tools, the best and average models succeed on only 41.8% and 21.7% (Table 3), so SAVIT is far from saturated with current tools. This hidden risk is also realistic (see W1, item 4).
5. Difference from agentic LLM safety work → Prior work tests whether text agents resist visible harm; SAVIT tests whether VLMs uncover hidden risk from images. In SAVIT, no text states the risk: the model must notice that safety-critical information is missing, inspect the image, turn visual evidence into searches, and use what it finds. We also score information acquisition and response appropriateness separately to show where agents fail. Appendix B details the comparison.
W3. The first metric is unfair to safe samples. ...
Thank you for the comment. Requiring the safety information on safe samples is fair and necessary: a model cannot know that a request is safe without checking, because its unsafe twin looks the same. Our revised metric also credits facts found anywhere in the trajectory, and all conclusions hold on unsafe samples alone. We address each concern below and include these analyses in the revised paper.
1. "A fact counts as acquired only if the final response states it." → We now also credit facts found in the trajectory. The revised metric credits each fact if it appears in the final response or anywhere in the model's tool results or reasoning. This raises safe-sample acquisition from 20.3% to 24.0% on average, and all conclusions remain unchanged (Table 3).
(Continued in the next comment.)
Response to Reviewer QJeN (3)
(Continued from the previous comment.)
2. "For safe samples the model only has to comply … and it may already know the rule." → Blind compliance is not reliable safety. In 63% of scenarios (165 of 263), the safe and unsafe variants have word-for-word identical requests and differ only in the image. The request alone therefore cannot establish whether assistance is safe. Our data show why this matters (pooled over the 14 models). When a model answers a safe sample appropriately without the safety information, it responds inappropriately to the unsafe twin of the same scenario 70% of the time. When it answers with the information, its responses to the twin are far better (54.5% vs. 29.9% appropriate; 40.2% vs. 12.8% joint success). Thus, safe compliance alone conceals substantial failures on the matching unsafe cases. A model that already knows the rule can simply state it, since every response includes a justification.
3. "They do not show how it still supports their conclusions." → All conclusions hold on unsafe samples alone, where the metric is uncontroversial.
| Unsafe samples only | Result |
|---|---|
| Joint success with tools | 21.1% on average; best 40.2% (Claude Opus 5) |
| Rank agreement with all-sample joint success | Kendall's τ = 0.89 (Spearman's ρ = 0.97) |
| Acquisition vs. response appropriateness | 28.8% vs. 38.9%: acquisition remains the bottleneck |
| Effect of tools on joint success | Improves all 14 models (13.7% → 21.1% on average) |
The acquisition requirement on safe samples therefore does not drive our conclusions; it ensures that compliance is informed rather than lucky.
Q1. Which tools does each sample actually need, and how often is each tool used? ...
Thank you. Appendix H.5 now includes these statistics and per-model numbers.
Tools each sample needs. By design, each sample requires zoom-in, web search, or both (Appendix D.1). For the 525 samples solved by at least one model, majority votes over tools used in each sample's successful runs give the split below.
| Both | Web search only | Zoom-in only |
|---|---|---|
| 46% | 38% | 14% |
Tool use across 14 models (Table 3, means and ranges across models)
| Tool | Calls per sample | Samples using it (%) |
|---|---|---|
| Web search | 1.18 (0.21–2.70) | 37.5 (12.2–80.8) |
| Page fetch | 0.06 (0.00–0.20) | 3.0 (0.0–9.9) |
| Zoom-in | 0.79 (0.01–1.53) | 40.9 (0.5–79.7) |
| Code interpreter | 0.35 (0.00–1.09) | 9.5 (0.1–39.1) |
Zoom-in is most widely used. More tool use correlates with higher acquisition (Spearman's ρ = 0.82). With the same tools, a research agent averages 6.0 searches and 3.7 page fetches per sample, showing that models under-use the tools (Appendix H.4).
Same tools for all instances. All models receive the same four tools for every instance, with identical descriptions, settings, and limits. Provider-native tools are disabled.
Q3. How would the acquisition result change on safe samples...
Thank you. High acquisition failure persists without the final-answer-only rule and on unsafe samples alone. Conditional results show why safe compliance alone cannot establish reliable safety knowledge. We add these analyses to Appendix H.6.
1. A broader metric changes the result only slightly. We credit facts in the final response, tool results, or available reasoning. Average acquisition failure falls from 76.0% to 73.4%, and safe-sample acquisition rises from 20.3% to 24.0%. Acquisition still fails on 71.2% of unsafe responses.
(Continued in the next comment.)
Response to Reviewer QJeN (4)
(Continued from the previous comment.)
2. Knowing the rule does not establish which visual case applies. In 63% of scenarios, safe and unsafe twins have identical requests, and only the image distinguishes them. A text-only probe tests prior rule knowledge, but cannot establish whether the model identifies the visual case and applies the rule. Our metric checks for task-specific safety information in the recorded interaction. We therefore examine conditional appropriateness and matched-twin behavior.
3. Conditional results show why compliance alone misses safety failures.
| Response appropriateness (%) | Acquired | Not acquired |
|---|---|---|
| Safe samples | 92.1 | 88.3 |
| Unsafe samples | 72.4 | 26.7 |
On unsafe samples, acquisition is associated with 45.7 percentage points higher response appropriateness. On safe samples, appropriateness stays high with or without the information, so compliance alone cannot reveal the model's safety knowledge.
Among appropriate safe responses, the same model answers the unsafe twin appropriately 54.5% of the time with acquisition on the safe sample, versus 29.9% without it. Apparently successful safe compliance thus conceals substantial failures on matching unsafe cases.
Q4. How was the data pipeline filtered in practice?
Thank you for the question. We report the full funnel below and add it to Appendix D.2. Human review is strict and rejects 58% of the candidates, whereas the final no-tool screening removes only 7.5% of samples, so most reviewed samples were already tool-dependent.
| Stage | Scenario pairs kept | Removed at this stage |
|---|---|---|
| Generated by LLMs | 720 | – |
| Manual review | 305 | 415 pairs (58%) rejected, minor issues revised |
| Image acquisition | 305 (799 samples) | none |
| Tool-dependency screening | 303 (739 samples) | only 60 of 799 samples (7.5%) |
Some variants use several images, which gives 799 samples.
Scenario diversity (Appendix J). Claude Opus 5.5, viewing every image, annotated each scenario (a set of safe and unsafe samples sharing one request) with its subtopic, task, and countries (CBRNE excluded for safety). Near-duplicates are scenarios whose request, safety information, and image are all near-identical in embedding space.
| Category | Scenarios | Subtopics | Near-dup. | Example subtopics |
|---|---|---|---|---|
| Criminal Activities | 29 | 24 | 1 | vehicle equipment legality, age limits for purchases, self-defense sprays |
| Intellectual Property | 40 | 16 | 0 | dataset licenses, font licenses, public-domain sound recordings |
| Privacy | 42 | 23 | 0 | marriage records, voter registers, photo location metadata |
| Unwanted Bias | 43 | 10 | 0 | gender stereotypes about women's careers, disability in hiring, religious items |
| Medical Harm | 41 | 26 | 1 | drug–diet interactions, food storage limits, venomous bites and stings |
| Risky Behavior | 30 | 18 | 1 | color taboos, bicycle rules, hand gestures |
| Misinformation | 38 | 24 | 0 | juice–drug interactions, Arctic operations, population origins |
The scenarios cover 141 subtopics (10–26 per category) and 76 countries and regions, with only 1% near-duplicates, and span diverse tasks (advice 46%, text editing 21%, writing 11%, planning 10%, how-to 9%).
In a blind comparison, the same annotator rated about 165 samples from each benchmark for the countries involved, whether real users could face the situation (plausible situation), and whether a real user could send the message and image (natural input), on a 5-point scale.
| SAVIT | MM-SafetyBench | SIUO | HoliSafe | |
|---|---|---|---|---|
| Countries covered | 44 | 2 | 28 | 3 |
| Plausible situation (% rated 4–5) | 97.0 | 48.8 | 35.5 | 54.8 |
| Natural input (% rated 4–5) | 80.6 | 0.0 | 36.1 | 42.8 |
| Requests stating user context (%) | 96.4 | 9.0 | 69.3 | 5.4 |
SAVIT covers far more countries than existing benchmarks and is judged realistic nearly twice as often as the next-best benchmark.
Response to Reviewer QJeN (1)
Dear Reviewer QJeN,
We appreciate your positive comments on our two-axis evaluation design. We have incorporated your suggestions into the revised paper (changes highlighted in red).
SAVIT's data are high quality: independently verified facts behind labels, tool-dependent samples even for frontier models, and diverse, realistic scenarios. We add this evidence to the paper.
1. LLM generation/manual review → Every label rests on verified facts. Two frontier models of different families generate scenarios via web search. We manually check every scenario and image against explicit criteria, keeping only 42% of pairs (Appendices D.2–D.3). Claude Code automatically retrieves the annotated facts for 95.1% of samples with the evaluated models' tools (Appendix H.4).
2. One small screening model → Additional GPT-5.5 screening. We do not rely on one small model: the same no-tool filter with GPT-5.5 removes only 48 more samples (6.5%). All main conclusions and model rankings remain unchanged on 691 samples (Appendix H.3, Table 6). No-tool acquisition is only 26.8% at best and 15.2% on average (Table 3).
3. Pipeline → Strict review retains tool-dependent samples.
Human review rejects 58% for major flaws and revises minor issues (Appendix D.2). No-tool screening removes only 60/799 samples (7.5%): most reviewed samples already require tools.
4. Diversity → Broad coverage and greater realism.
Method. Claude Opus 5.5 views every image and labels each scenario (safe/unsafe samples sharing one request) by subtopic, task, label-deciding knowledge, and countries (CBRNE excluded for safety). Near-duplicates have near-identical embeddings of requests, safety information, and images.
SAVIT covers 141 subtopics (10–26 per category) and 76 countries and regions, with only 3 of 303 near-duplicate scenarios (1%):
Tasks span advice (46%), text editing (21%), writing (11%), planning (10%), and how-to (9%). Label-deciding knowledge spans laws, item identification, facts, licenses, health, and privacy (11–17% each).
Blind comparison. We mix ~165 samples per benchmark (SAVIT, MM-SafetyBench, SIUO, HoliSafe), hiding sources. The same annotator:
SAVIT tests safety across far more countries' laws and customs. Situations and inputs are judged realistic nearly twice as often as the next-best benchmark; rates remain high on all SAVIT samples (93.2% and 85.3%). By design, requests state user context (96.4%) and seek ordinary help whose safety depends on local laws, licenses, and drug interactions.
5. Tools needed → Web search, zoom-in, or both. Our instruction and review enforce this (Appendices D.1–D.2). Successful runs use both tools for 46% of solved samples, web search only for 38%, and zoom-in only for 14%. We will release this per-sample information.
6. Vague labels or scores tied to one fact → Labels rest on verifiable facts, and finding them is what SAVIT measures. Changing facts require tools: SAVIT tests checking current facts rather than memory. Our agent re-finds these facts on the live web; versioned releases include source and retrieval metadata (Appendix B). The judge accepts equivalent wording and facts from any source in tool results or reasoning. Rankings remain stable across judges (Kendall's τ ≥ 0.89).