- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a query-based black-box attack that uses only the target LLM's textual outputs, without requiring logits, confidence scores, or auxiliary models. The key observation is that LLMs can provide useful binary comparisons between candidate inputs, even when their absolute confidence estimates are poorly calibrated. Based on this observation, the authors develop a hill-climbing approach and evaluate it on adversarial examples, prompt injections, and jailbreaks. The paper also investigates the effects of model capability, transfer-based initialization, and stronger search strategies.
Critical strengths
- The paper presents a simple and interesting approach to black-box optimization. Using binary comparisons to extract optimization signals directly from the target model is intuitive and requires neither numerical feedback nor auxiliary models.
- The experiments cover diverse attack scenarios and reveal interesting empirical findings. Beyond demonstrating effectiveness across multiple models and tasks, the cross-model experiments suggest a relationship between model capability, preference quality, and attack effectiveness, offering insights into potential vulnerabilities in stronger LLMs.
Critical weaknesses
- The comparison with existing black-box attacks is insufficient. The paper largely excludes established methods such as PAIR and TAP because they use auxiliary models, although these methods operate under the same target-access constraints. However, it is common and easy to use auxiliary models in practice, and evaluation under typical auxiliary models is more significant. Given that the authors also use surrogate CLIP models for transfer initialization, comparisons with stronger existing attacks under comparable query budgets would better establish the practical significance of the proposed method.
- Lack of straightforward baselines and incremental improvements in effectiveness. The paper lacks comparisons with simple response-only baselines, such as random search or success/failure-guided optimization, under matched query budgets. Moreover, improvements over Transfer-only are marginal on several vision-LLMs, making it unclear how much additional benefit the proposed feedback mechanism provides in these settings.
Questions for authors
- For vision-language adversarial examples, attack success is evaluated using a fixed binary classification prompt (e.g., "Does this image contain a dog?"). How sensitive are the reported ASRs to this evaluation prompt?
- How sensitive is the binary preference signal to query wording, candidate ordering, and sampling randomness?
- Defense implications: Given the observed capability–vulnerability relationship, what insights does the proposed method offer for designing effective defenses? In particular, can models be trained to resist providing attack-useful preference feedback without substantially compromising their general comparison and reasoning capabilities?