acceptodds
Under review as a conference paper at ICLR 2027

Quality Hijacking: Semantic-Aware Typographic Attacks on Image Quality Assessment

Abstract

Vision-language model-based image quality assessment methods (VLM-IQA) have shown strong capability in evaluating complex and out-of-distribution images, yet remain vulnerable to typographic attacks. Attacking VLM-IQA requires adversarial text to effectively manipulate quality-related reasoning while remaining visually consistent with the underlying image. Existing typographic attacks typically rely on generic adversarial texts (ATs) and heuristic placement strategies, overlooking task-related factors and local image degradation, which limits both attack effectiveness and visual plausibility. To address these limitations, we propose Q-Hack, a black-box framework for typographic attacks against VLM-IQA models. Q-Hack adopts a coarse-to-fine strategy to jointly optimize AT content and placement. In the coarse stage, it explores three IQA-related attack factors to generate candidate texts, and naturally places them in suitable regions based on scene semantics, local degradation, and depth. In the fine stage, it exploits target-model feedback to distill effective semantic patterns and refine both text content and placement for stronger attacks. We compare Q-Hack with seven state-of-the-art baselines on three widely used IQA datasets, where it demonstrates clear advantages across attack effectiveness, perceptual fidelity, and scene consistency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image quality assessment, typographic attack, vision-language model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_W76YOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Q-Hack, a black-box typographic attack framework targeting vision-language model-based image quality assessment (VLM-IQA). The method adopts a coarse-to-fine strategy that first generates three types of quality-related adversarial text and identifies suitable placement regions using semantic, degradation, and depth information. It then iteratively refines text-region pairs based on target-model feedback to improve attack effectiveness. Experiments on three IQA datasets and four VLM-IQA models demonstrate that Q-Hack effectively manipulates predicted quality scores while maintaining perceptual fidelity and scene consistency.

Critical strengths
  1. This paper provides clear task-specific motivation by identifying three categories of quality-related adversarial text and investigating how text types and placement in degraded regions affect attack effectiveness.
  2. The proposed framework integrates semantic, degradation, and depth information for natural text placement, achieving strong attack effectiveness and perceptual fidelity across multiple datasets and VLM-IQA models.
Critical weaknesses
  1. The methodological novelty is limited, as joint text-placement planning, naturalistic rendering, and black-box feedback optimization have been explored in prior works [1–3]; the main contribution lies in task-specific adaptation and integration.
  2. The observed advantage of placing adversarial text in degraded regions is mainly established through artificially introduced localized distortions, with limited direct evidence of its underlying mechanism or generalizability to complex real-world degradation patterns.
  3. This paper lacks an independent ablation of IQAGR and comparisons under matched query budgets, making it difficult to distinguish methodological improvements from gains due to additional optimization.
  4. Although Eq. (2) formulates a multi-objective optimization problem, the refinement stage primarily relies on target IQA scores for candidate selection, leaving the roles of scene consistency and perceptual fidelity in the actual optimization process unclear.

[1] Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, and Qing Guo. SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments.

[2] Ziyuan Luo, Xinyu Chen, Qi Song, and Renjie Wan. Naturalistic Typographic Attacks on VLM-Based Image Quality Assessment.

[3] Shihong Liu, Samuel Yu, Zhiqiu Lin, Deepak Pathak, and Deva Ramanan. Language Models as Black-Box Optimizers for Vision-Language Models.

Questions for authors
  1. Can the authors clarify the key methodological differences between Q-Hack and prior works [1–3], particularly regarding text-region joint optimization, beyond task-specific adaptation?
  2. Can the authors provide additional experiments on real-world localized distortions to validate the effectiveness of degradation-aware placement, especially compared with placing text in non-degraded or degradation-mismatched regions?
  3. Can the authors provide an ablation study without IQAGR and compare Q-Hack with existing methods under matched query budgets to better isolate the contribution of feedback-guided refinement?
  4. How are the scene consistency and perceptual fidelity terms in Eq. (2) incorporated into the actual optimization process, and how are the weighting parameters μ and λ determined?

[1] Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, and Qing Guo. SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments.

[2] Ziyuan Luo, Xinyu Chen, Qi Song, and Renjie Wan. Naturalistic Typographic Attacks on VLM-Based Image Quality Assessment.

[3] Shihong Liu, Samuel Yu, Zhiqiu Lin, Deepak Pathak, and Deva Ramanan. Language Models as Black-Box Optimizers for Vision-Language Models.

4 comments

Q1 & W1. Technical novelty

We acknowledge that several technical components of Q-Hack have been explored in prior works. However, our contribution lies in how these techniques are integrated and adapted to address the specific challenges of VLM-IQA, rather than introducing entirely new components. The key methodological differences are summarized as: 1) compared to [1], Q-Hack establishes a correspondence between text semantics and local quality degradations, enabling quality-aware text-region matching; 2) compared to [2], Q-Hack jointly considers quality degradation, scene semantics, and depth geometry to identify regions that are both attack-relevant and visually plausible; 3) compared to [3], Q-Hack extends feedback-driven optimization to text-region pairs, enabling the target IQA model's feedback to jointly refine adversarial text content and spatial placement.

Q2 & W2. Observation of degradation-aware placement

To validate the generalizability of this observation, we conduct additional experiments on real-world distortions. Specifically, we use Q-Ground to identify distorted and undistorted regions in images, along with the corresponding distortion types. We then consider three text placement settings: 1) placing attack text in undistorted regions (Table 5); 2) placing attack text in distorted regions with mismatched distortion types (Table 5); and 3) placing attack text in distorted regions with matched distortion types (Table 5). Please refer to Section 4.3 for detailed experimental settings. The results show that the third setting leads to the best attack performance, confirming that degradation-aware placement can generalize to real-world image distortions. It indicates that VLM-IQA assesses visual quality based on local distortions in images; placing semantically matched attack text in these distorted regions helps highlight such distortions.

Q3 & W3. Ablations on IQAGR

We evaluate IQAGR with different rounds (0, 2, 4, 6, 8, 10, 12, and 14), where 0 indicates that IQAGR is disabled. The experimental results in Fig. 6(d) in the revised manuscript show that 1) Q-Hack still achieves stronger attack performance than baselines even without IQAGR; and 2) ASR increases with the IQAGR rounds and gradually converges after 10 rounds.

We also evaluate a minimum-cost configuration of Q-Hack (denoted as Ours*) in Tables 3 (Section 4.2) and 7 (Appendix C.3). Ours* requires only 3 queries, which is comparable to advanced typographic attacks such as SceneTAP (1 query) and NTA (1 query). In this case, Ours* still achieves better attack performance compared to baselines under comparable query budgets.

Q4 & W4. Issues on Eq. (2)

We apologize for the confusion caused by the inappropriate formulation of Eq. (2) in the original submission. Ideally, an effective typographic attack against IQA should consider three aspects: attack effectiveness, scene consistency, and perceptual fidelity. However, Q-Hack addresses these aspects through different mechanisms rather than jointly optimizing them. Specifically, Q-Hack employs semantic-aware region selection and LVTR-based rendering to promote scene consistency, whose effectiveness is supported by objective metrics (i.e., TSIS) and our newly added human subjective evaluation (Section 4.2). Regarding fidelity, Q-Hack selects the smallest suitable region to limit the size of the embedded text, thereby reducing its impact on the original image quality. The MAE and DISTS results further demonstrate its superior fidelity preservation compared with existing methods. Therefore, Q-Hack explicitly optimizes only attack effectiveness during the refinement stage.

To better reflect this distinction, we have revised Eq. (2) by reformulating scene consistency and perceptual fidelity as constraints rather than explicit optimization objectives. The revised Eq. (2) formulates the desired attack problem, while Q-Hack provides a practical solution that explicitly optimizes attack effectiveness and implicitly promotes scene consistency and perceptual fidelity through its design. Accordingly, the thresholds ϵ1\epsilon_1 and ϵ2\epsilon_2 conceptually characterize acceptable levels of scene consistency and perceptual fidelity, rather than serving as numerical constraints explicitly enforced during refinement.

Thank you again for your thoughtful comments and constructive feedback. We sincerely hope that our response has adequately addressed your concerns.

As the discussion period nears its deadline, we would appreciate it if you could share whether our responses have sufficiently clarified the issues you raised or if any concerns remain.

Thank you very much for your time and consideration!

Thank you for your response. It has addressed my questions. That said, I still find it difficult to fully assess the overall quality of the manuscript, so I have lowered my confidence.

Thank you for taking the time to review our revised responses and for acknowledging that we have addressed your questions. We understand that you may still have some reservations regarding the overall quality of the manuscript, and we sincerely appreciate your thoughtful feedback and the time you have devoted to reviewing our work.

Reviewer_Tp6qOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Q-Hack, a black-box framework for typographic attacks against VLM IQA models. It generates image-conditioned text and places it using semantic, degradation, and depth cues. Experiments on LIVE Challenge, AGIQA-1K, and TID2013 evaluate four VLM-IQA models against the proposed attack and seven baselines. Evaluation covers attack success and perceptual fidelity. Q-Hack achieves the highest attack success in most settings and generally provides better trade-off between fidelity and attack success. Ablation studies examine the contribution of each component. Experiments with two defenses assess the attack’s strength.

Critical strengths
  1. IQA-Specific Insights. The paper shows that the most effective text category varies across image–model pairs and that placing text in degraded regions typically results in greater score increases. These findings provide an IQA-specific basis for jointly optimizing text content and placement.
  2. Broad Empirical Evaluation. Q-Hack is tested on three datasets and four VLM-IQA models, with comparisons against seven other attack methods. The authors also conduct ablation studies and evaluate the attack against two defenses. Q-Hack provides higher attack success in most settings, while maintaining a better trade-off between attack effectiveness and perceptual fidelity. This supports the relevance of the results across multiple datasets and VLM-IQA models
Critical weaknesses
  1. The claims about perceptual fidelity and scene consistency are not supported by a human study. In particular, TSIS is assessed by the same auxiliary VLM used to generate the attack, which may bias the evaluation. A human perceptual comparison would strengthen the evidence for visual naturalness.
  2. The results are reported without confidence intervals. This makes it difficult to assess whether the observed differences between attack methods are statistically reliable.
Questions for authors
  1. Did the authors conduct an human evaluation of text-scene naturalness? Since Qwen3-VL is used both to generate and refine the attacks and to score TSIS. This would provide an independent check of whether TSIS aligns with human judgments of naturalness.
  2. Сould the authors include confidence intervals in all result tables? This would help readers assess the variability and statistical reliability of the reported differences, especially where Q-Hack’s ASR is close to that of a baseline.
  3. Could the authors report the total number of target-model queries and the end-to-end runtime per image for Q-Hack and the compared baselines? This would clarify its practical cost relative to existing attacks.
  4. Could the authors clarify how the text’s appearance, particularly its font, color, size, and style, is determined? The paper states that LVTR uses the surrounding background as context, but it is unclear whether these visual attributes are selected or optimized by Q-Hack.
3 comments

Q1 & W1. Human evaluation

We conduct a human subjective evaluation to assess scene consistency in Section 4.2 of the revised manuscript. The results confirm that the text embedded by Q-Hack is less likely to be perceived as anomalous by human observers than that embedded by baseline methods. Therefore, TSIS scores align well with human subjective judgments.

Q2 & W2. Confidence interval

Thank you for the valuable comment. Given the randomness of the proposed attack (e.g., varying the random seeds), we repeated each experiment five times and reported only the mean results in the original submission. Following the suggestion, we have now added the corresponding confidence intervals to all tables, calculated using Student's t-distribution based on the results of five independent runs.

Q3. Efficiency and cost

Following the reviewer’s suggestion, we report the average number of queries to the target model (Avg. Queries), the end-to-end runtime per image (RPI), and the computational token cost (TC) in Table 3. Experimental results show that 1) When setting K=5K=5 and n=10n=10 (Ours), Q-Hack requires 25 queries to the target model, significantly fewer than existing query-based attacks (Black-UAP: 431 queries; AttackIQA: 10001 queries); 2) When setting K=1K=1 and n=0n=0 (Ours*), Q-Hack requires only 3 queries, which is comparable to advanced typographic attacks such as SceneTAP (1 query) and NTA (1 query). Although reducing the query budget decreases Q-Hack’s attack effectiveness, it still achieves better attack performance compared to baselines under comparable query budgets (Table 7).

Q4. Impact of text appearance

Text appearance is entirely determined by LVTR, and Q-Hack does not separately optimize these text attributes. In our work, attack effectiveness is primarily determined by text placement and content. LVTR is used solely to ensure that the embedded text appears natural and visually consistent with the surrounding scene. Both the objective evaluation metrics (TSIS) and the newly added human subjective evaluation (Section 4.2) provide evidence of LVTR's effectiveness in generating natural-looking text. To further investigate the impact of text appearance on attack effectiveness, we have added a new experiment in Section 4.3 of the revised manuscript, where we fix the text content and placement determined by Q-Hack while varying the appearance of the embedded text. Experimental results in Fig. 6(b) and Fig. 6(c) show that 1) changes in font, color, and style have relatively minor effects; 2) excessively small fonts weaken attack performance due to reduced visibility.

Thank you for your answers; they fully address my questions. I recommend acceptance.

Thank you very much for your positive feedback and recommendation. We sincerely appreciate your time and consideration of our responses. We are glad that our clarifications have addressed your concerns.

Reviewer_GAWCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes Q-Hack, a typographic attack method against VLM-IQA. It generates different types of AT and selects text placement based on image semantics, quality-degraded regions, and depth information. It also uses score feedback to iteratively improve the attack. The authors conduct extensive experiments and use ablation studies and defense experiments to further evaluate the method.

Critical strengths

The paper studies typographic attacks specifically for image quality assessment, rather than directly applying general attacks against vision-language models. This makes the research meaningful.

Critical weaknesses
  1. Scene consistency is mainly evaluated using scores from an auxiliary VLM. Without human evaluation, it is difficult to fully support the claims that the adversarial text looks natural and is hard to notice.
  2. The attack process involves multiple auxiliary models, candidate generation, and multiple rounds of scoring. However, the paper does not report the runtime per image or model query costs. In addition, the paper does not analyze false rejection rates for clean images in sufficient detail.
Questions for authors
  1. How are quality-degraded regions defined in the paper? What criteria are used to select regions from the quality map? How much would the attack performance change if these criteria were changed?
  2. Please report the runtime per image, the number of queries to the target model, and the computational cost of Q-Hack. Does Q-Hack have any efficiency advantages over the baseline methods?
  3. Can scene consistency be evaluated through independent human evaluation?
  4. When the original image already contains different types of text, does this affect the attack performance? Can the text in the original image interact with the inserted adversarial text and affect the results? Have the authors conducted a separate analysis of images that already contain text?
3 comments

Q1. Definition and selection of degraded regions, and sensitivity to selection criteria

Q-Ground is a model fine-tuned on a quality-region segmentation dataset (annotated by humans and LLMs) using pixel-level and mask-level losses. It can localize and segment quality-degraded regions in input images while identifying their corresponding distortion types.

We select the “minimum suitable region” from masked distortion regions. We determine a “Suitable region” based on three aspects. 1) Text-region consistency: the selected region should correspond to the degradation type targeted by the current adversarial text (defined by Eq. (4)). 2) Text accommodation: the region's shape and size should be sufficient to accommodate the candidate text. 3) Visual region coherence: The selected region should form a spatially contiguous and visually homogeneous area, preferably belonging to a single object or surface with consistent texture and appearance. Regions spanning multiple objects, materials, or distinct textures are avoided to prevent unnatural text integration. This criterion is assessed using an auxiliary VLM guided by a predefined prompt. When multiple regions satisfy these criteria, we select the one with the smallest area to ensure perceptual fidelity.

To evaluate the sensitivity of the region selection strategy, we conducted five controlled experiments: 1) with vs. without the quality map generated by Q-Ground (Table 5); 2) matched vs. mismatched adversarial text and distortion types (Table 5); 3) different auxiliary VLMs (Table 8); 4) different auxiliary VLM prompts (Table 9); and 5) smallest vs. random vs. largest region selection (Table 10). Please refer to the revised submission for detailed experimental settings. Experimental results show that 1) Q-Hack's attack effectiveness is sensitive to the use of the quality map, the matching between adversarial text and distortion types, and the prompts without text accommodation, but is relatively insensitive to the choice of auxiliary VLM and the smallest-region selection strategy; 2) Q-Hack’s scene consistency decreases when the “visual region coherence” instruction is removed from the prompts.

Q2 & W2. Efficiency of Q-Hack and false rejection rates (FRR) for clean inputs

We report the runtime per image (RPI), the average number of queries to the target model (Avg. Queries), and the computational token cost (TC) in Table 3 in the revised manuscript. Experimental results show that 1) When setting K=5K=5 and n=10n=10 (Ours), Q-Hack requires 25 queries to the target model, significantly fewer than existing query-based attacks (Black-UAP: 431 queries; AttackIQA: 10001 queries); 2) When setting K=1K=1 and n=0n=0 (Ours*), Q-Hack requires only 3 queries, which is comparable to advanced typographic attacks such as SceneTAP (1 query) and NTA (1 query).Although reducing the query budget decreases Q-Hack’s attack effectiveness, it still achieves better attack performance compared to baselines under comparable query budgets (Table 7).

We report FRR (below 15%) of target models on clean images in Section 4.2. Please refer to Appendix C.1 for the calculation of FRR.

Q3 & W1. Human evaluation

We conduct a human subjective evaluation to assess scene consistency in Section 4.2 of the revised manuscript. The results confirm that the text embedded by Q-Hack is less likely to be perceived as anomalous by human observers than that embedded by baseline methods.

Q4. Impact of pre-existing texts

To investigate the impact of pre-existing text in images on attack effectiveness, we collect four sets from tested datasets: 1) Set-1 contains no text; 2) Set-2 contains only numeric characters (0–9); 3) Set-3 contains only alphabetic text (A–Z, a–z); and 4) Set-4 contains both numbers and alphabetic characters. Experimental results in Fig. 6(a) of the revised manuscript show that pre-existing texts weaken all tested attacks (Center, SceneTAP, NTA, and Q-Hack). Nevertheless, Q-Hack still achieves the best performance across all four sets.

Thanks for the response. The authors have addressed all my concerns, especially the analysis of pre-existing text and the practical efficiency. I will raise my score to 3.

We sincerely appreciate your constructive feedback during the initial review. We are also grateful for your thoughtful reassessment of our responses and recognition of our additional efforts.

Comments on the paper

1 comment

To all reviewers:

We sincerely thank all reviewers for their valuable feedback and for recognizing the merits of our work:

We propose Q-Hack, a novel quality-aware typographic adversarial attack against VLM-IQA models. By jointly considering local quality degradation, scene semantics, and depth geometry, Q-Hack generates effective and visually plausible adversarial text placements.

In our revision, we have made the following updates:

  1. Region Selection Strategy: We provide a more detailed explanation of the suitable region selection criteria, including text-region consistency, text accommodation, and visual region coherence in Section 3.3. We also conduct additional experiments to evaluate the sensitivity of the selection strategy in Appendix C.6.

  2. Experimental Evaluation: We add human subjective evaluations to validate scene consistency and investigate the impact of pre-existing text on attack effectiveness in Section 4.2. We add confidence intervals to all experimental tables. We also report the FRR of target models on clean images in Section 4.2.

  3. Efficiency Analysis: We report the average number of queries, runtime per image, and computational token cost in Table 3.

  4. Additional Ablation Studies: We conduct further experiments on IQAGR optimization rounds, text appearance, and degradation-aware placement in Section 4.3.

  5. Problem Formulation: We revise Eq. (2) to formulate scene consistency and perceptual fidelity as constraints rather than explicit optimization objectives, better reflecting the actual design of Q-Hack.

We hope these updates address your concerns and further strengthen the contributions of our work. Thank you again for your thoughtful reviews and support!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.