acceptodds
Under review as a conference paper at ICLR 2027

TIPGuard: A Unified Suite for T2I Prompt Guardrails with Customizable Safety Policies

Abstract

Guardrail models are a cornerstone of trust for deploying AI at scale. In text-to-image (T2I) scenarios, they face a fundamental challenge: mapping abstract safety norms into accurate, fine-grained judgments of open-ended visual content while adapting to evolving safety policies. However, existing approaches often rely on costly data collection and human annotation, and their safeguards remain tied to predefined safety policies. To address these challenges, we introduce TipGuard, a unified framework spanning data synthesis, model training, and fine-grained evaluation. At its core is an almost fully automated synthesis pipeline that produces semantically rich and diverse safety supervision for realistic T2I requests with minimal human involvement. Using this pipeline, we construct, to the best of our knowledge, the largest T2I guardrail dataset, with 90% agreement between synthetic annotations and human judgments. We further train a series of guardrail models tailored to different deployment scenarios. These models achieve state-of-the-art performance across multiple safety benchmarks and demonstrate surprisingly strong generalization to previously unseen safety policies. We further introduce TipGuardBench, a benchmark for multi-category, fine-grained safety judgments, and use it to systematically assess the capabilities and limitations of existing guardrail models. Together, these findings demonstrate the feasibility of building policy-customizable T2I guardrails with largely automated supervision. By unifying data synthesis, model training, and evaluation around explicit safety policies, TipGuard provides a foundation for developing guardrails whose safety criteria can be revised as deployment requirements evolve.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Text-to-Image Generation, Prompt Safety, Safety Guardrails, Policy Customization, Content Moderation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_t6G8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces TIPGuard for text-to-image prompt moderation. It develops a safety policy framework comprising 528 rules and approximately 2.36 million training examples, uses supervised fine-tuning (SFT) to train moderation models supporting custom policies and fixed categories, and provides fine-grained evaluation datasets.

Critical strengths
  1. The problem has practical value, with attention to visual scene context and changes in safety policies.
  2. The data construction process is fairly systematic, combining automated synthesis, hierarchical annotation, and human feedback.
  3. Defining both prohibited and permitted cases helps clarify safety boundaries and reduce false positives.
  4. The method performs well on the authors' own benchmarks, while external evaluations also suggest some potential for practical applications.
Critical weaknesses
  1. Since the user prompts are generated by language models, how is prompt diversity ensured across the two million examples? Could the diversity of the dataset be quantified?
  2. Why are the baselines in Table 2 not included in the comparisons in Table 3? ImageGuard is an output-side detector, whereas TIPGuard is an input-side detector. Is this comparison fair?
  3. The evaluation of generalization is insufficient. Can the model understand policies that are explicitly held out from training?
  4. For the same user input, can the model correctly adjust its judgment when the applicable policy changes?
Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Weakness 1 - Weakness 4

W1. Dataset diversity and quantitative evaluation

Thank you for suggesting a quantitative assessment of dataset diversity. Our diversity and scalability come from Concept Extraction → Visual Grounding → Keyword-to-Prompt Generation. Each concept expands into visual scenarios: the harmful branch varies subjects, actions, appearance, and risk-related context, while the safe branch varies subjects, settings, and visual styles. Scenario keywords are then converted into complete prompts.

Exact-text deduplication across the Visual Grounding outputs yields:

ComponentHarmful branchSafe branch
Concepts30,685216,637
Visual scenarios846,9551,527,783
Visual keywords3,838,7922,768,947

We further quantify lexical diversity using Distinct-2 (Li et al., 2016), comparing the TIPGuard-2M and T2ISafety datasets. With identical tokenization and a fixed 5,000-token budget, we sample 900 candidate prompts per dataset per run across five different random seeds, reporting the mean and standard deviation.

DatasetDistinct-2 (%, mean ± standard deviation)
TIPGuard-2M71.11 ± 1.59
T2ISafety60.85 ± 3.11

Under the same token budget, TIPGuard-2M achieves a 10.26-percentage-point higher Distinct-2 score. Therefore, the staged construction process and the quantitative Distinct-2 metric jointly support the diversity of the dataset.

W2. Evaluation protocols and input-side versus output-side comparison

  1. The baselines in Table 2 evaluate prompt moderation before generation, whereas ImageGuard's original experiment in Table 3 evaluates generated images. This work focuses on input-side guards and does not include image generation, so we did not extend the Table 2 baselines to output-side evaluation on generated images.

  2. Regarding the evaluation protocol and comparability, both methods start from the same input prompts and use the same F1 calculation method, making their results comparable. TIPGuard uses the official prompts and their original category labels, with the original policy definitions supplied as inference input and no additional training; ImageGuard evaluates images generated from the corresponding prompts, and its scores are taken from the original paper. The two methods identify risks at the input and output stages, respectively, and this comparison shows that a substantial portion of risks can be identified before image generation.

W3–W4. Generalization to unseen policies and adaptation to policy changes

  1. Apart from TIPGuardBench, all other evaluation benchmarks we use adopt safety policies not seen during training, testing the model's understanding of and generalization to external policies. An unseen policy refers to specific policy definitions not used during training, rather than requiring entirely different risk-category names. Different safety policies can share broad risk categories while differing in their scope, exceptions, and criteria for permitting or blocking a request. At evaluation, external policies are supplied as input to TIPGuard without additional training. The evaluation results in the paper support the model's generalization and adaptation across different external benchmarks and safety policies.

  2. Yes, TIPGuard can adjust its judgments for the same user input according to changes in the applicable safety policy. We added a category-removal experiment on all unsafe TIPGuardBench samples, removing each sample's original top-level category while retaining all others and their definitions. Prompts and model parameters remained fixed. We evaluated safety decisions and category assignments, with 50 randomly selected samples reannotated by humans under their modified policies.

    MetricResultScope
    Classified as safe after category removal53.27%All samples
    Assigned to the removed category0%All samples
    Model–human agreement under modified policies92.0%50 random samples

    Typical category transitions observed in the experiment include Hate ↔ Discrimination, Suicide & Self-Harm → Violence & Gore, Illegal Activities & Crime ↔ Dangerous Goods, and Sexual Content [Sexual Violence] → Violence & Gore.

    Correct policy adaptation means reassessing requests under the remaining policy, rather than allowing every request after its original category is removed. A request can be allowed when its sole applicable restriction is removed; if it still involves another restricted risk, it should be assigned to a remaining applicable category. These results show that TIPGuard can adjust its judgment of the same input in response to policy changes.

Thanks! My concerns are addressed and I will raise my score.

Reviewer_jvKYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents TIPGuard, an input-side guardrail that classifies T2I prompts under explicit safety policies. It introduces a hierarchical safety policy, a large synthetic training dataset, and a human-verified benchmark, then trains policy-conditioned and fixed-label moderation guardrail models. Experiments show strong prompt-classification performance and demonstrate that changing policy definitions can alter moderation decisions without retraining.

Critical strengths
  1. The work represents a substantial engineering effort, including building a large labeled dataset, developing a detailed policy hierarchy, and training several models for prompt synthesis and safety moderation.
  2. The construction pipeline is reasoanble and well motivated. Extracting safety concepts, turning them into concrete visual scenes, generating T2I prompts, and refining annotations through human feedback provides a practical approach to producing supervision at scale.
  3. The evaluation includes external datasets and human-feedback ablations, helping assess transfer beyond the authors’ own benchmark and showing the value of refining policy definitions and synthetic annotations.
Critical weaknesses
  1. The practical benefit for image-generation safety remains unverified. Text-only input guardrail is reaonsable, but the experiments measure agreement with prompt-level policy labels rather than safety outcomes in generated images. They do not establish whether deploying TIPGuard reduces unsafe images while preserving access for benign requests. An end-to-end evaluation across image generators would provide a stronger connection between the reported classification gains and the paper’s practical safety motivation.
  2. Human validation of the synthetic training data is insufficiently documented. The authors check sampled automatic annotations and refine policies until agreement exceeds 90%, but do not report further details such as sample sizes, annotator details. These omissions make the annotation-quality claim and the amount of human involvement difficult to assess.
  3. The construction of Tipguard-Reasoner supervision is unclear. The paper does not adequately explain how reasoning traces are generated and verified.
Questions for authors

Please see weakness part.

2 comments
AuthorsOpenReview ↗

Weakness 1 - Weakness 3

W1. Practical safety benefits

Thank you for highlighting practical safety benefits. We respond from two perspectives: guardrail responsibilities and the balance between safety and utility:

  1. The two types of guards in T2I safety have different responsibilities. A prompt guard operates before image generation, assessing whether the visual content described by a request violates the safety policy, rejecting policy-violating requests early and avoiding unnecessary generation costs; an image guard operates after generation, directly checking whether the generated images are safe. The two are complementary rather than interchangeable.

    This work focuses on prompt guarding, not on predicting the safety of generated outputs. When TIPGuard makes its decision, no image has yet been generated, so it does not need to directly observe output risks introduced by generator-specific behavior or stochastic sampling. We agree that a semantically compliant prompt may still produce an unsafe image, but this does not mean that the request itself violates the policy; such risks need to be handled by output-side image guards.

  2. We evaluated TIPGuard on OVERT, measuring both over-refusal on benign prompts and rejection of unsafe prompts.

    SystemOver-refusal rate (%) ↓Unsafe rejection rate (%) ↑
    SD-3.5-Large2.003.00
    DALL-E-3-API18.5057.20
    Imagen-329.1048.60
    FLUX1.1-Pro35.9054.60
    DALL-E-3-Web51.7076.30
    TIPGuard (ours)20.2884.43

    Our Guard achieves an over-refusal rate of 20.28% and an unsafe rejection rate of 84.43%, the highest unsafe rejection rate among the compared methods. Compared with DALL-E-3-API, our over-refusal rate is 1.78 percentage points higher, while our unsafe rejection rate is 27.23 percentage points higher. Furthermore, our Guard achieves both lower over-refusal and higher unsafe rejection than Imagen-3, FLUX1.1-Pro, and DALL-E-3-Web, thereby Pareto-dominating these three baselines.

    These results provide a dedicated assessment of over-refusal and demonstrate that our Guard extends the empirical safety–utility Pareto frontier of the compared baselines, rather than achieving stronger unsafe rejection merely through indiscriminately rejecting more requests.

W2. Human-validation scale, annotator assignments, and feedback procedure

Thank you for your attention to the details of human validation. The annotation team comprises experts with experience in LLM safety evaluation, safety taxonomy development, or generative-model output annotation, and all annotators receive training on the safety policy and four-level label specifications before undertaking the task.

Within each classification scope, approximately 300 category-balanced samples are randomly drawn per round. Each sample is assigned to three experts for independent annotation, without access to model predictions, explanations, or other annotators' judgments, and majority voting determines the human reference label. Model–human disagreements are then analyzed to revise the policy and reannotate the data; after each revision, a fresh random sample is drawn for independent human validation. Each scope typically undergoes approximately three rounds, with iteration ending when agreement between the model and human reference labels exceeds 90%.

We will add a Human Annotation Protocol appendix to the paper, detailing annotator qualifications, sampling, independent annotation and reference-label determination, disagreement analysis, and iterative validation.

W3. Generation, validation, and training use of reasoning supervision

Training CoT comes from DeepSeek-V4-Flash with thinking mode enabled. During annotation, the model not only outputs the matched risk category but also explains why that category applies and why the other candidate categories do not, followed by a final summary. We organize these outputs into CoT supervision for reasoning mode.

CoT is refined through human-feedback-driven policy revision. Experts inspect the CoT of samples where model and human judgments disagree, analyze the discrepancies, and revise the safety policy, guiding the model to regenerate CoT that better aligns with the basis for human judgments. The detailed procedure is provided in the Human Annotation Protocol appendix to be added to the paper.

Thanks for the author's response. It addresses most of my concerns, especially the results on an over-refusal benchmark and the details on construction. I hope these experiement results and dataset construction detailes can be clarified in the revised manuscript. And I'd like to keep my positive ratings.

Reviewer_416MOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • A prompt-side T2I guardrail suite with four parts:
    • a 4-level policy with 528 rules (64 of them permit rules)
    • TIPGuard-2M, built by turning HH-RLHF concepts into visual keywords, then into prompts with a benign-trained generator; DeepSeek-V4-Flash labels them, with a human-feedback loop
    • Qwen3.5-9B guards: Reasoner (policy in the prompt; direct or reasoning mode) and Lite (fixed labels)
    • TIPGuardBench (3,056 prompts)
  • Claims: state-of-the-art on TIPGuardBench (+7.7 binary F1 over GPT-OSS-SafeGuard-120B, Tab. 2), transfer to unseen policies (T2ISafety, Tab. 3), and parity on real prompts (Unsafe Diffusion, Tab. 4).
Critical strengths
  • Good, timely idea. Policy-customizable prompt moderation for T2I is practically relevant, and explicit permit rules mark the boundaries.
  • Human-annotation round trip: the policy is refined until model–human agreement reaches ≥90%, with large ablation gains (Tab. 5).
  • The generator is trained only on benign data and uses verifiable rewards to get around refusals. Detailed generator, guard and synthesis templates are provided (App. B–D).
  • Multi-granularity evaluation is diagnostic: existing guards collapse at fine levels (LlamaGuard3 L4 = 5.8, Tab. 2).
Critical weaknesses

Missing controls and baselines

  • The base Qwen3.5-9B with the same policy is absent from Tab. 2–4. Gains cannot be separated from the backbone, and guards have been shown to underperform their own base model on unseen policies [1].
  • No frontier model is evaluated in context with the policy. The most important missing one is the labeller, DeepSeek-V4-Flash: it supplies the automatic (human-refined) labels, so it is the key teacher reference.
  • The T2I guards (GuardT2I, Latent Guard, DiffGuard, PromptGuard) and policy guards (DynaGuard, GuardReasoner, LlavaGuard) are cited but not evaluated. These are exactly the two lines the paper claims to bridge. Tab. 3–4 also lack the text-guard baselines used in Tab. 2.

Benign prompt → unsafe outcome

  • Robustness to benign prompts that produce unsafe outputs is not demonstrated. A prompt guard cannot observe generator-specific or stochastic failures, and the paper does not discuss this.
    • Direct testbeds are ART [8] and Adversarial Nibbler [9]; I2P and ImplicitBench [10] are related.
    • The paper gives no protocol for when to defer to output-side checks [21] and has no limitations section.
  • No adversarial evaluation, despite SneakyPrompt being cited as motivation; see [12–14].

Evaluation validity

  • TIPGuardBench is drawn from the TIPGuard-2M pipeline, with the same generator, labeller and policy, so its distributional independence is limited.
    • Exact exclusion is stated, but semantic train–benchmark decontamination is not.
    • Each blocking rule has only 2 unsafe examples. No CIs or seeds are reported.
  • The human-feedback evidence is confounded:
    • Tab. 5 trains "w/o HF" on the original taxonomy and scores it against refined-policy labels, mixing annotation quality with the change in target taxonomy.
    • The 90% agreement in the abstract is the stopping criterion. No fresh-sample estimate after the final refinement is reported.
  • "Unseen policy" is overstated:
    • The T2ISafety taxonomy was a policy-construction source (App. A).
    • On the genuinely unseen Privacy domain the results are mixed: direct mode trails on PF and PID (−0.144 / −0.177), reasoning mode wins PID but loses PF and IPV (Tab. 3).
    • The baseline scores appear to be T2ISafety's image-classification F1. ImageGuard is an image evaluator, and TIPGuard's prompt-level metric and protocol are unspecified.
  • The Fig. 5 definition rewriting is error-guided on the same benchmark it is re-scored on [16]. The margin over T2I-RiskyPrompt is +0.6 pts, and Explicit Pornography drops −10.5 (Tab. 11).
  • Tab. 4 is inconsistent under standard binary-metric definitions, and rounding cannot explain the gap:
    • TIPGuard's F1 is not the harmonic mean of its P/R (.845 / .865 vs. reported .82 / .84).
    • Acc/P/R imply about 57% unsafe prompts for TIPGuard vs. about 30% for every baseline.
    • Averaging conventions, evaluation set and whether baselines were rerun are all unspecified, so the F1 superiority cannot currently be validated.
  • No dedicated over-refusal evaluation [11] and no separate breakdown on the 1,000 boundary cases.
  • The policy-variation strategy ("varied category definitions") is unspecified, and its effectiveness against policy overfitting is not established. No perturbation or category-removal evaluation is reported, unlike [2, 7].

Model claims

  • The "low-latency" Lite is never measured. It uses the same 9B backbone, so any saving comes only from omitting policy tokens.
    • The trade-off is unclear: Reasoner-direct wins on binary F1, while Lite L2 wins L1 and L2 macro-F1 (Tab. 2).
  • No consistent guidance on when to use reasoning mode. Direct mode wins all of Tab. 2 and the HF-refined Tab. 5, but reasoning mode wins Tab. 4 and IA/PID in Tab. 3 [15]. The provenance of the reasoning traces is unspecified.
  • How L3/L4 policies are serialized or routed at inference is not described. The only length stated is the 4,096-token training limit.
  • Output is single-label, so multiple independent violations across branches (e.g. Violence + Hate) cannot be enumerated.

Data and novelty

  • All prompts come from one generator. There is no style or diversity analysis, and no breakdown of the difficulty of the 1.8M safe prompts or of the boundary cases.
  • The 91.6% "transfer" figure is the downstream BLOCK rate, not a measure of generation quality.
  • "Largest dataset" is claimed without a comparison, e.g. to CoPro (226k, in the cited Latent Guard), CoProV2 [17] or the survey [18].
  • Individual components have precedents:
    • policy-driven synthetic guard data with block/permit rules [3, 4]
    • hierarchical rule-in-prompt labelling [5] and tiered policy taxonomies [19]
    • image-side policy-following guards [6, 7]
    • Positioning of the combination against these is missing, and the claim that the two lines "remain disconnected" should be qualified.

Minor

  • Generator SFT is 1 epoch in Sec. 4.1 but 3 in App. B.4.
  • The 630k BLOCK prompts in the synthesis pool (Tab. 9) are not reconciled with the 538k unsafe prompts in the final dataset (Tab. 1).
  • Fig. 3: the Self-Harm reason duplicates the Disturbing example.
  • Fig. 4 labels differ from App. C.
  • Policy sources include blog posts (cf. [20]).
  • No release statement.

References

  • [1] Liu et al., Domain Generalizable AI Guardrails with Augmented Policy Training (APT), ACL 2026
  • [2] Piao et al., Towards Policy-Adaptive Image Guardrail: Benchmark and Method (SafeEditBench / SafeGuard-VL), CVPR 2026
  • [3] Sharma et al., Constitutional Classifiers: Defending against Universal Jailbreaks…, arXiv 2501.18837, 2025
  • [4] Mazza & Levi, BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate, arXiv 2604.25203, 2026
  • [5] Li et al., Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling (Hi-Guard), KDD 2025
  • [6] Xu et al., SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability, arXiv 2510.23960, 2025
  • [7] Yu et al., ProGuard: Towards Proactive Multimodal Safeguard, arXiv 2512.23573, 2025
  • [8] Li et al., ART: Automatic Red-teaming for T2I Models to Protect Benign Users, NeurIPS 2024
  • [9] Quaye et al., Adversarial Nibbler, FAccT 2024
  • [10] Yang et al., Towards Implicit Prompt for Text-to-Image Models (ImplicitBench), ICML 2024
  • [11] Cheng et al., OVERT: A Benchmark for Over-Refusal Evaluation on T2I Models, NeurIPS 2025
  • [12] Yang et al., MMA-Diffusion, CVPR 2024
  • [13] Tsai et al., Ring-A-Bell!, ICLR 2024
  • [14] Huang et al., Perception-guided Jailbreak (PGJ), AAAI 2025
  • [15] Sreedhar et al., Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models, EMNLP 2025
  • [16] Palla et al., Policy-as-Prompt: Rethinking Content Moderation in the Age of LLMs, FAccT 2025
  • [17] Liu et al., AlignGuard (CoProV2), ICCV 2025
  • [18] Bavalatti et al., A Systematic Review of Open Datasets Used in T2I Gen-AI Safety, IEEE Access 2025
  • [19] Zeng et al., AIR-Bench 2024, 2024
  • [20] Riccio et al., Exploring the Boundaries of Content Moderation in T2I Generation, ECCV-W 2024
  • [21] Liu et al., Wukong: NSFW Detection in T2I Systems, KDD 2025 / Yang et al., In-Generation NSFW Detection, arXiv 2508.03006

An automated paper review flagged the following works

  • Unseen-policy guards: GSPR, Roblox Guard 1.0, RSafe, CoSA, Aegis 2.0 (the basis of NemoGuard), Granite Guardian.
  • Synthetic and fine-grained guard data: BingoGuard, Markov et al. (AAAI 2023), SALAD-Bench.
  • Visual guards: Llama Guard 3 Vision, ShieldGemma 2, GuardReasoner-VL, VModA, UnsafeBench.
  • Attacks: JPA, Multimodal Pragmatic Jailbreak, Jiang et al. 2025.
  • T2I prompt moderation: AEIOU, VALOR, PurifyGen, SafeGen / NSFW-56k, Safe-CLIP / ViSU, VLSU.
  • Other: LPG, ThinkGuard, R²-Guard, SafeWatch, OmniGuard.
Questions for authors
  • What do Qwen3.5-9B base, DeepSeek-V4-Flash and one frontier model score on Tab. 2–4 when given the same policy?
  • How do the guards perform on benign prompts that yield unsafe outputs (ART, Adversarial Nibbler)?
  • The T2I-vs-generic policy mismatch is shown only through two anecdotes. How many TIPGuardBench labels flip under a generic policy?
  • Tab. 3/4: what is TIPGuard's metric and averaging convention, what are the splits and modality, and were baselines rerun or copied? Please explain the P/R/F1/Acc inconsistency.
  • How are L3/L4 policies serialized or routed at inference, and what is the inference context length? Who generated the reasoning traces?
  • Can Fig. 5 be repeated with definitions rewritten on a held-out split? Can you report latency for Lite vs. Reasoner?
  • Beyond exact exclusion, how was the benchmark semantically decontaminated from training? Please report results on the 1,000 boundary cases separately.
  • What is the human-feedback loop's scale (samples, rounds, inter-annotator agreement)? Were samples refreshed between iterations, and what is the agreement on a fresh sample after the final revision?
  • How many Minor Sexual Content prompts exist in TIPGuard-2M and the benchmark, and what legal review and access controls apply? Please give annotator details, IRB status and the safeguards used.
  • What are the language split and the performance on Chinese for the final dataset and benchmark? Will the data, models and generator be released?
6 comments
AuthorsOpenReview ↗

Weakness 1 - Weakness 3.1

W1. Missing controls and baselines

We evaluated Qwen3.5-9B (TIPGuard’s backbone) and DeepSeek-V4-Flash (our annotation model) on TIPGuardBench using the same policy definitions and evaluation protocol as TIPGuard-Reasoner. We also evaluated DynaGuard, GuardReasoner, DiffGuard, and GuardT2I, using native binary decisions where available and adapting DynaGuard for hierarchical classification.

ModelBinary Macro-F1L1 Macro-F1L2 Macro-F1L3 Macro-F1L4 Macro-F1
Qwen3.5-9B80.355.749.442.535.1
DeepSeek-V4-Flash79.563.655.449.340.4
DynaGuard-8B80.942.625.615.010.3
GuardReasoner-8B (Binary Only)74.9————
DiffGuard (Binary Only)41.2————
GuardT2I (Binary Only)23.1————
TIPGuard-Reasoner direct94.677.770.764.853.3
TIPGuard-Reasoner reasoning93.274.467.659.145.7

Both direct and reasoning modes outperform all baselines on their supported metrics, demonstrating gains beyond supplying the policy to the original models.

Latent Guard requires policy-specific concept mapping and threshold calibration, while LlavaGuard and PromptGuard target image moderation and generation-time safety, respectively, outside TIPGuard’s prompt-level classification scope.

W2–Q2. Benign prompt → unsafe outcome

Thank you for raising this issue. We believe your comment reflects a misunderstanding of the roles and functions of prompt guards and image guards in T2I safety, and we clarify their respective responsibilities below:

  1. The two types of guards in T2I safety have different responsibilities. A prompt guard operates before image generation, assessing whether the visual content described by a request violates the safety policy, rejecting policy-violating requests early and avoiding unnecessary generation costs; an image guard operates after generation, directly checking whether the generated images are safe. The two are complementary rather than interchangeable.

  2. This work focuses on prompt guarding, not on predicting the safety of generated outputs. When TIPGuard makes its decision, no image has yet been generated, so it does not need to directly observe output risks introduced by generator-specific behavior or stochastic sampling. We agree that a semantically compliant prompt may still produce an unsafe image, but this does not mean that the request itself violates the policy; such risks need to be handled by output-side image guards.

Therefore, TIPGuard determines whether a request itself violates the policy, rather than guaranteeing the safety of generated outputs; unsafe images generated from benign prompts should be detected and blocked by output-side image guards.

W3. Evaluation validity

W3.1–Q7. Benchmark design and validity

Thank you for your concern about evaluation validity. We respond from three perspectives: construction objectives, benchmark selection and coverage, and external generalization evaluation:

  1. TIPGuardBench and the training set share a construction pipeline but serve different purposes: TIPGuard-2M provides large-scale training supervision, while TIPGuardBench enables fine-grained evaluation of policy understanding. This approach has precedent: Llama Guard, Section 3.3 likewise splits a shared annotated data pool into fine-tuning and evaluation sets. Thus, sharing a construction pipeline while serving different purposes is a reasonable evaluation design.

  2. Beyond this construction pipeline, TIPGuardBench undergoes human selection and verification. First, pool-level near-duplicate filtering and manual selection for unique keywords and distinct scenarios reduce repetitive content; selected benchmark records are then excluded from the candidate training data. Second, human selection and verification ensure coverage of all rules and policy-compliant boundary cases, combining rule coverage and scenario diversity for a more appropriate test-data distribution.

  3. In addition, we evaluate generalization on external benchmarks, including T2ISafety and T2I-RiskyPrompt, with different data sources and safety policies. These results provide cross-source and cross-policy evidence beyond TIPGuardBench and its shared construction pipeline.

In summary, different construction objectives, human selection and verification with rule coverage, and complementary external evaluations support our evaluation design; external benchmark performance also supports the model’s strong generalization.

Regarding CIs and seeds, we would appreciate further clarification: which stage's uncertainty is being referred to here, and for which experiments and metrics should confidence intervals and random seeds be reported?

AuthorsOpenReview ↗

Weakness 3.2 - Weakness 3.4

W3.2–Q8. Human-feedback evaluation and validation

Thank you for your comments. We address the evaluation setup in Table 5 and the agreement validation procedure as follows:

  1. In Table 5, w/o HF is trained on original-policy annotations and evaluated against the original policy and its labels; HF is trained on refined-policy annotations and evaluated against the refined policy and its labels. Thus, the training–evaluation policy mismatch described in the review does not occur. This experiment evaluates the downstream benefits of the complete human-feedback-driven process of policy refinement and reannotation.

  2. Agreement is validated on newly sampled prompts with fresh independent human annotation. After each policy revision and reannotation, we draw a new category-balanced sample, with each prompt independently labeled by three experts. Agreement is measured against their majority-vote labels; each scope is finalized when agreement exceeds 90%, so the final round evaluates the latest policy on newly sampled data.

We will add a Human Annotation Protocol appendix detailing annotator qualifications, sampling, independent annotation, reference-label determination, disagreement analysis, and iterative validation.

W3.3. Transfer to unseen policy definitions

Thank you for your comments. We believe this concern stems from differing interpretations of “unseen policy” granularity. We clarify the definition, generalization results, and evaluation setup below:

  1. An unseen policy refers to definitions not used during training, not necessarily new category names. Policies may share risk categories but differ in scope, exceptions, and permit/block criteria. T2ISafety served as background on risk domains, but its policy was not included in our training data. At evaluation, TIPGuard receives its original definitions without additional training, testing its ability to apply unseen policy definitions.

  2. All evaluation categories in T2ISafety constitute OOD evaluation under an unseen policy, not just Privacy. Their definitions and decision boundaries were not used during guardrail training, so results across the benchmark reflect generalization to the external policy. TIPGuard-Reasoner direct achieves an Overall score of 0.805, with strong performance across multiple OOD categories, individual weaknesses do not imply poor overall generalization, the overall result and performance across unseen categories provide direct evidence of policy generalization.

  3. Regarding the evaluation protocol and comparability, both methods start from the same input prompts and use the same F1 calculation method, making their results comparable. TIPGuard uses the official prompts and their original category labels, with the original policy definitions supplied as inference input and no additional training; ImageGuard evaluates images generated from the corresponding prompts, and its scores are taken from the original paper. This comparison shows that substantial risk detection is achievable before image generation.

In summary, we evaluate generalization to external policy definitions, not merely new category names, and its application to input-side risk identification.

W3.4–Q6.1. Feedback-driven category definition rewriting

This experiment demonstrates the ability to adjust a model's moderation behavior through policy text without retraining; the key result is a 5.1-percentage-point accuracy improvement for the same model, rather than merely a 0.6-point margin over the baseline; its effectiveness should be assessed using overall performance and the full category-level results, rather than focusing solely on a single category that declines.

AuthorsOpenReview ↗

Weakness 3.5 - Weakness 3.7

W3.5–Q4. Table 4: metrics and corrections

We sincerely thank the reviewer for carefully examining the metrics in Table 4 and for this valuable observation.

For consistency with Table 2’s multiclass evaluation, we originally macro-averaged Precision, Recall, and F1 over the safe and unsafe classes. This convention was omitted from Table 4. Macro-F1 averages class-specific F1 scores, rather than taking the harmonic mean of macro-Precision and macro-Recall; this explains the discrepancy, independently of rounding.

Using the original saved per-sample predictions, we recounted TP, FP, FN, and TN and recalculated Precision, Recall, and F1 with unsafe as the positive class, following the benchmark’s binary-metric convention. Predictions and evaluation samples are unchanged; no new model run was performed. Baseline scores were taken directly from the original paper, not rerun.

MethodAccuracyPrecisionRecallF1
Safety filter0.750.590.520.55
Q160.700.490.730.59
Fine-tuned Q160.880.770.830.80
Multi-headed SC0.900.870.780.82
TIPGuard-Reasoner direct0.82 (−0.08)1.00 (+0.13)0.73 (−0.05)0.84 (+0.02)
TIPGuard-Reasoner reasoning0.84 (−0.06)1.00 (+0.13)0.76 (−0.02)0.86 (+0.04)

TIPGuard-Reasoner outperforms all listed baselines in F1 in both direct and reasoning modes, with reasoning mode achieving the highest F1 of 0.86.

W3.6. Dedicated over-refusal evaluation

We evaluated TIPGuard on OVERT, measuring both over-refusal on benign prompts and rejection of unsafe prompts.

SystemOver-refusal rate (%) ↓Unsafe rejection rate (%) ↑
SD-3.5-Large2.003.00
DALL-E-3-API18.5057.20
Imagen-329.1048.60
FLUX1.1-Pro35.9054.60
DALL-E-3-Web51.7076.30
TIPGuard (ours)20.2884.43

Our Guard achieves an over-refusal rate of 20.28% and the highest unsafe rejection rate of 84.43%. Compared with DALL-E-3-API, our over-refusal rate is 1.78 percentage points higher, while our unsafe rejection rate is 27.23 percentage points higher. Furthermore, our Guard achieves both lower over-refusal and higher unsafe rejection than Imagen-3, FLUX1.1-Pro, and DALL-E-3-Web, thereby Pareto-dominating these three baselines.

These results provide a dedicated assessment of over-refusal and demonstrate that our Guard extends the empirical safety–utility Pareto frontier of the compared baselines, rather than achieving stronger unsafe rejection merely through indiscriminately rejecting more requests.

W3.7. Training-time policy variation and policy adaptation

Thank you for your comments. We explain training-time policy diversity and adaptation to policy changes as follows:

  1. Training-time policy diversity comes from different hierarchical levels and category branches. Each unsafe prompt is paired with four level-specific policies along its annotated path, containing different candidate categories and definitions. The full hierarchy yields 165 policy templates (L1+L2+L3+L4=1+10+34+120L1 + L2 + L3 + L4 = 1 + 10 + 34 + 120). This trains the model to classify the same prompt under the current policy rather than assign a fixed label from the prompt alone.

  2. We added a category-removal experiment to directly test whether the model adjusts its judgments in response to policy changes. For all unsafe TIPGuardBench samples, we remove each sample’s original top-level category while retaining all others and their definitions. Prompts and model parameters remain fixed. We evaluate safety decisions and category assignments, with 50 randomly selected samples reannotated by humans under their modified policies.

    MetricResultScope
    Classified as safe after category removal53.27%All samples
    Assigned to the removed category0%All samples
    Model–human agreement under modified policies92.0%50 random samples

    Typical category transitions observed in the experiment include Hate ↔ Discrimination, Suicide & Self-Harm → Violence & Gore, Illegal Activities & Crime ↔ Dangerous Goods, and Sexual Content [Sexual Violence] → Violence & Gore.

    Correct policy adaptation means reassessing requests under the remaining policy, rather than allowing every request after its original category is removed. Requests can be allowed if their sole restriction is removed, or reclassified if another restriction remains, with the human audit supporting the correctness of these adjustments.

In summary, hierarchical variation provides diverse training policies, while the category-removal experiment demonstrates policy-conditioned decisions beyond a fixed prompt-to-category mapping.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.