- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces FVH Bench, a benchmark of ~700 prompts evaluating if image generators produce artifacts that facilitate harmful actions or deception without necessarily depicting overtly harmful content. It evaluates 11 generators, assesses generated artifacts using automated and human judgments, and examines existing safety detectors and a proposed "function aware" detection approach.
Critical strengths
- The paper is clearly written, with an understandable taxonomy and illustrative examples.
- It highlights a somewhat plausible limitation of image safety evaluations: harmful functionality can arise from diagrams, documents, and interfaces rather than overtly harmful imagery.
- The evaluation spans 11 generators and multiple safety detectors, with human validation of automated scoring and a text–image comparison.
- Overall I think the writing quality and presentation is good, and the paper seems fairly well done. I would not be opposed to its acceptance, but I question the overall utility of a dataset of this scale with a threat model this specific.
Critical weaknesses
- Refusal itself might be a result of ambiguity in each image's models implicit or explicit safety policy, rather than an unambiguous safety failure.
- The operational threat model is underdeveloped. Ratings of apparent harmful utility do not establish whether artifacts would successfully deceive recipients or enable harmful actions in realistic settings.
- The benchmark is relatively small and synthetically constructed, leaving uncertainty about its coverage of realistic misuse.
- The evaluation also does not establish whether text-only prompt screening would adequately address the identified risks. I suppose that it could make sense for image detection removed from AI Image generators, assuming text-based safety filters have been removed.
Questions for authors
- What assumptions define the threat model, incuding the attacker, recipient, distribution channel, and contextual information needed for an artifact to be credible or actionable?
- How effectively would a text-only safety classifier detect harmful intent in the generation prompts? What additional risks require per image assessment?
5 comments
Response to Reviewer 546e (Part2)
W3: The benchmark is small and synthetically constructed, raising questions about its coverage of realistic misuse.
R3: Thank you for this concern. FVH-Bench provides reproducible tests of functional risks across models, functions, and domains, rather than estimating real-world misuse frequencies. Its utility rests on three aspects:
-
Coverage is organized by function and domain, rather than a single narrow scenario. Section 3.2 describes 669 requests spanning five functions and ten domains, covering direct harm, fraud and platform abuse, and social harm. This supports comparisons of the same function across domains and different functions within a domain.
-
Human review checks whether synthetic requests express clear harmful objectives and interpretable risk mechanisms. Section 3.3 and the construction appendix describe iterative checks followed by five independent reviewers assessing each candidate's harmful objective, function/domain assignments, clarity, and consistency. Retention requires at least four reviewers to approve all criteria. The full construction process retains 669 of 750 initial candidates. Appendix examples illustrate these mechanisms through fabricated records, misleading health infographics, and simulated interfaces. These checks support task validity and category assignments, not representativeness of real-world misuse frequencies.
-
The benchmark supports comparative diagnosis beyond demonstrating that risks exist. Evaluating eleven generators on the same requests enables comparisons of refusal, harm realization, and functional support. Section 5.2 identifies differences across functions: mean HAU is 3.57 for evidence fabrication and 4.09 for persuasive packaging. Such differences characterize functional risks and inform safeguard evaluation.
FVH-Bench turns functional visual risks into structured, reproducible evaluation tasks. As stated in the Discussion, its distribution does not represent real-world request frequencies; our conclusions concern the covered scenarios, without claiming exhaustive coverage.
W4/Q2: How effectively can text-only screening detect harmful intent in generation prompts, and what additional risks require image-level assessment?
R4: Thank you for this important question. Text screening is an important pre-generation safeguard, but identifying harmful intent in a request and assessing the harmful support actually provided by an output are two different tasks. The value of image assessment does not depend on removing the generator's text filters. We report an existing text-only evaluation and clarify this distinction below.
- The evaluated text-only detector does not cover all benchmark requests. We evaluated Llama Guard 4 (12B, BF16) on all 669 original prompts, without images, sample-specific function/domain labels, or prior scores. All requests received valid decisions:
| Harmful visual function | Prompts | Flagged unsafe | Unsafe rate |
|---|---|---|---|
| F1: Action enablement | 199 | 89 | 44.72% |
| F2: Evidence fabrication | 139 | 42 | 30.22% |
| F3: Interface simulation | 71 | 39 | 54.93% |
| F4: Authority mimicry | 186 | 99 | 53.23% |
| F5: Persuasive packaging | 74 | 43 | 58.11% |
| Overall | 669 | 312 | 46.64% |
The detector flags 46.64% as unsafe and classifies 53.36% as safe. This is its unsafe flag rate on FVH-Bench, not accuracy or an upper bound on text-screening performance. Nor does every unflagged request necessarily violate its native policy. Existing text screening therefore cannot be assumed to cover all these requests.
-
Actual outputs cannot be inferred solely from the request, even when harmful intent is recognized. The same request may yield harmful content, a safe alternative, or an artifact limited by omissions, contradictions, or warnings. Text screening asks whether a request should be allowed; image assessment asks what was generated and how it supports harm. In Case F2.1, the false water-quality certificate presents a compliance claim, but contradictory measurements and conclusions limit its evidentiary utility. This requires inspecting the generated content and relationships. THR and HAU thus provide output-level information beyond intent classification.
-
Image assessment also applies when the original prompt is unavailable. Platforms or recipients receiving shared screenshots, documents, or infographics may lack the generation request and must assess the image itself. This reflects different information available at generation and distribution, not an assumption that text filters have been removed.
Text screening and image assessment therefore serve distinct purposes: input protection and output risk assessment. The text-only results reveal coverage gaps in one existing detector; image evaluation assesses realized harm and functional support. We do not equate the flag rate with additional blocking gains from a combined pipeline.
Dear Reviewer 546e,
Thank you again for your thoughtful review. As the discussion period is nearing its end, we wanted to follow up on our response. If you have a chance to take a look, we would appreciate hearing whether our clarifications address your concerns. We would be happy to answer any remaining questions. Thank you for your time and consideration.
Response to Reviewer 546e (Part1)
Dear Reviewer 546e,
Thank you for your thoughtful review and recognition of our clear presentation, taxonomy, and examples, and our evaluation across generators and detectors, human validation, and text–image comparison.
W1: Differences and ambiguity in model safety policies may affect refusal behavior, so low refusal rates do not necessarily constitute unambiguous safety failures.
R1: Thank you for highlighting policy differences. Our conclusions draw on both refusal behavior and functional harm in outputs; not every non-refusal is a violation of a model's own policy.
Policy compliance and functional risk coverage are distinct. Models may adopt different boundaries for documents, interfaces, or misleading information. A functional risk outside a policy's scope represents a coverage gap; harmful outputs within its scope may indicate insufficient recognition or enforcement. Policy differences affect the interpretation of refusal rates but do not eliminate the risks exhibited by the outputs.
Generated outputs provide evidence beyond refusal rates alone. Section 4.2 reports an average refusal rate of 2.45% across eleven models and a mean of model-level THR averages of 4.46, with eight models averaging above 4. Together with HAU, these results show that models rarely refuse and frequently produce artifacts that visibly realize harmful targets and provide functional support. We identify safeguard gaps for FVH-Bench risks, rather than infer policy violations from refusal behavior alone.
These findings support including functional visual risks in safety evaluation and safeguard design, as current safeguards do not adequately cover or recognize them.
W2/Q1: The threat model needs clearer assumptions about the attacker, recipient, distribution channel, and context needed for credibility or actionability. Apparent harmful utility does not establish successful deception or harmful action in realistic settings.
R2: Thank you for this question. Our threat model concerns a requester using an image generator to produce artifacts whose information organization, evidentiary presentation, or credibility cues support harmful actions or deception. We assess the functional support provided by these artifacts under the following assumptions:
Credibility and actionability depend on visible relationships relevant to the specified function, not realistic appearance alone. For documents, text, data, and validation cues must provide coherent apparent evidence for the target claim. For interfaces, fields, states, and messages must jointly support the targeted misunderstanding. For action-enabling diagrams, structure and information relationships must help users understand the specified action. Document formatting, institutional symbols, or polished layouts alone do not establish a harmful function.
Existing cases illustrate the distinction between functional support and actual success. In appendix Case F2.1, the false water-quality certificate presents a compliance claim, but contradictory data and conclusions limit its evidentiary utility (HAU: 2.67). Case F3.2 depicts a coherent platform state and messages supporting a false impression of an interaction (HAU: 4.67). This assessment neither requires a static screenshot to have a working backend nor establishes that a recipient would be deceived. Scores reflect the artifact's support and visible limitations.
Our evaluation addresses the stage from a harmful request to a usable visual artifact. The Discussion and evaluation prompts distinguish visible harm realization and apparent functional support from realized harm. FVH-Bench identifies when generators provide materials supporting harmful purposes and compares that support; artifact scores are not treated as the success rate of an end-to-end attack.