- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes CrossAnchor, which guides sparse edits within text prompts using image anchors and an ensemble of dual encoders. Optimization requires no target-model feedback, and the deployed input remains text-only. Evaluation covers six targets, two guardrails, and three judges. On a selected cohort of 161 prompts, optimized edits substantially outperform random replacements at high sparsity. A nine-attempt cascade achieves 71.4–97.5% success under Qwen3Guard and the primary GPT-4o judge.
Critical strengths
-
A relevant evaluation of combined defenses. The paper evaluates both input-guardrail evasion and downstream model behavior, exposing failures that component-level evaluation can overlook. The separation between image-based optimization and text-only deployment is practically interesting.
-
Useful paired controls and sensitivity analysis. Optimized and random-fill variants share prompts and editable positions. At 80% sparsity, optimization improves attack success by 14.3–64.0 percentage points across six targets under the primary judge. Additional guardrails, secondary judges, and targets excluded from cohort selection help establish the scope and robustness of the finding.
Critical weaknesses
-
The image anchor’s contribution remains insufficiently isolated. A matched text-anchor baseline is essential to distinguish the benefit of cross-modal anchoring from semantic reconstruction after corruption. Table 4 also changes ensemble size, similarity scope, and search budget together, so its 1.1-point average improvement provides limited evidence about the individual design choices.
-
The headline effectiveness is conditional on sample selection and retry budget. The cohort retains 161 of 1,000 candidates whose image anchors already succeed against all four selection targets. Original-text success is already 54.0–95.7% before guardrail gating (Tables 10–15). Furthermore, the nine-attempt cascade is compared primarily with a single original-text attempt. Evaluation on the original candidate pool, initially refused cases, and matched-budget baselines would clarify generality and the contribution beyond repeated attempts.
-
Success-label validity needs stronger evidence. Appendix D counts factual information related to the harmful topic as potential success, which may include benign engagement with the topic. Across all evaluated responses, pooled success is 49.3% under GPT-4o and 31.2% under either secondary judge (Table 27). The shared rubric and substantial disagreement motivate a human audit focused on assistance toward the original harmful intent.
Questions for authors
-
How does image anchoring compare with text anchoring when the ensemble, editable positions, scope, and optimization budget are held constant?
-
Can you report results on the original candidate pool and initially refused prompts, together with random-fill and competitive text-attack baselines under equal attempt budgets?
-
Can you provide a blinded human audit of representative successes and judge disagreements to validate intent-specific harmful assistance?
Detailed response to reviews have been provided to the reviewers.
We have provided a detailed response with explanations to all the reviews.