- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper attacks the confidence signal that deployed vision-language models (VLMs) use to decide whether to accept or defer an answer. A white-box attacker perturbs only the input image (L∞ ≤ 8/255) and is required to keep the model's generated answer byte-identical. The attack lowers confidence on correct answers and raises it on wrong ones. Across four VLMs (InternVL3.5-2B, Molmo2-4B, LLaVA-OneVision-7B, Gemma-3-12B), three benchmarks (GQA, VQAv2, POPE) and seven confidence readouts, the authors report:
- with correctness labels, the attack drives 83 of 84 readout-cell combinations below chance; without labels, six of the seven readouts fall below chance in every cell;
- 62% of the inversion survives when the full next-token distribution is held within 0.2 nats (KL) of the clean one;
- attacked AUROC splits exactly into a same-answer term and a cross-answer term, which explains the one surviving case (Molmo/POPE);
- adversarially trained probes, a robust vision encoder (FARE) and a separate verifier VLM all fail to give a confidence signal that is both informative and robust under a white-box attacker;
- a gate tuned to a 10% false-accept rate admits up to 89.8% of the wrong answers it would otherwise reject, and, at each model's natural error rate, accepts answers less accurate than having no gate.
Critical strengths
- The constraint is strict and verified. Every answer token must stay the top prediction during the attack, and the final 8-bit image is re-decoded and must reproduce the answer byte for byte (Section 3).
- The main attack result is well checked. Within the tested models and readouts, it holds across three attack seeds, at half the budget (4/255), and when each readout is scored only on the attack aimed at it (Section 4).
- The KL-bound experiment is a convincing check. It rules out the explanation that the attack only exploits low-probability tokens (Section 4).
- The same-answer / cross-answer split is exact and useful. It explains why Molmo/POPE survives, and the long-answer experiment isolates the cross-answer mechanism (Sections 4–5).
- The deployment analysis is concrete. The gate experiment, including the combined attack that first plants a wrong answer and then gets it past the gate (77% on LLaVA, 88% on Molmo), shows practical impact (Section 4).
- The reporting is honest. Success thresholds were fixed before the experiments, and limitations are stated clearly.
Critical weaknesses
1. The novelty is modest.
- The paper itself states that a confidence attack with the prediction held fixed "is therefore settled" by prior work on image classifiers and text language models (Section 1), and names ACE as the closest predecessor (Section 2).
- The attack is standard PGD, and all seven readouts are existing methods.
- The main conceptual contribution is therefore extending a known attack to open-ended VLM generation and showing that common defenses do not stop it.
- The question that would add the most depth is left open: why one model's confidence can be moved and another's cannot. On long answers, the attack inverts the readouts on Gemma and Molmo but not on InternVL, and none of the properties the authors examine (reasoning training, answer length, backbone, size, strength of the correctness signal, input-to-logit sensitivity) predicts this split (Section 5). The Conclusion lists it as the open question.
- Even a partial answer would raise the contribution well beyond extending a known attack, for example a hypothesis tested on the existing models, or a controlled comparison between a model whose confidence moves and one whose confidence resists (see Question 6).
2. Model coverage is incomplete.
- The Qwen-VL family, one of the most widely used open VLM families at this scale, is not tested.
- The largest model is 12B, and each family is represented by a single model, so the effect of model size cannot be assessed. If compute is the constraint, adding one or two size variants within an existing family (for example, a smaller and a larger model of the same family) would show whether the effect changes with scale.
3. The readouts cover fewer method families than seven suggests.
- Four of the seven readouts (P(IK), SAPLMA, FULL, COUPLED) are the same MLP architecture on hidden states, and CCPS-D is a differentiable surrogate of CCPS (Appendix A.1).
- Sampling-based uncertainty (agreement across several sampled answers) is not attacked (Section 2, Limitations).
- Verbalized confidence appears only in the universal-perturbation control, not in the main attack (Section 3).
- A gate that combines several readouts is not tested. The paper's results suggest images aimed at one readout already move others, but a joint attack on a combined gate is not evaluated.
The claim that no readout "stays both informative and reliable" (Conclusion) should be limited to single-pass readouts, or these families should be tested.
4. The defenses tested are limited.
- The model itself is never adversarially fine-tuned; the robustness-versus-informativeness trade-off is shown only for probes on frozen hidden states (Section 5, Limitations).
- Only one robust encoder (FARE) is tested, on one model (LLaVA).
- The verifier is described as "a second, independently trained model" (Section 6), but it is one of the other three studied VLMs answering a yes/no prompt (Appendix H.1). A model trained or fine-tuned for verification, a larger model, or several verifiers combined might behave differently.
- Randomized defenses and adaptive purification are not tested; JPEG purification is evaluated only against a non-adaptive attacker and on three models (Limitations).
5. The threat model narrows practical relevance.
- The attack is white-box and per-item. It does not transfer across models, and no universal perturbation works (Limitations, Appendix I.1).
- JPEG at quality 50 removes between a quarter and two thirds of the manipulable range (Limitations).
- The threat therefore applies mainly to open-weight deployments where the attacker controls the image and has gradient access. The paper should state this scope more prominently in the abstract and introduction.
Questions for authors
- Do the results hold for Qwen-VL models and for models larger than 12B?
- How do sampling-based and verbalized confidence behave under the same answer-preserving attack?
- Does adversarially fine-tuning the full model, rather than a probe on frozen states, change the robustness-versus-informativeness trade-off?
- Images aimed at one readout already move others (for example, TokProb inverts without being targeted). Under a joint attack on a gate that combines several readouts, or several verifiers, does the combined gate fail as completely as a single readout does?
- Would a verifier trained specifically for answer verification behave differently from the prompted VLMs used here?
- Do you have any hypothesis for what makes a model's confidence more or less movable under an answer-preserving perturbation?
16 comments
Author Response to Reviewer TwMU (2/2)
Continued from our previous message.
| Your concern | Our response |
|---|---|
| TwMU.6 “The model itself is never adversarially fine-tuned”; “the robustness-versus-informativeness trade-off is shown only for probes on frozen hidden states”; “Does adversarially fine-tuning the full model, rather than a probe on frozen states, change the robustness-versus-informativeness trade-off?” | We added joint adversarial adaptation of the answering model and the probe, using LoRA across the vision encoder, projector, and language model, and found that it improves the trade-off. Attacked AUROC rises from to – across two initializations while clean confidence stays informative. |
| TwMU.7 “Only one robust encoder (FARE) is tested, on one model (LLaVA).” | We added TeCoA-4, a second robust vision encoder trained with a different objective, and found the same pattern as with FARE. Both make confidence harder to manipulate, most clearly at small budgets, but neither prevents inversion at the headline budget. |
| TwMU.8 “A model trained or fine-tuned for verification […] or several verifiers combined might behave differently”; “Would a verifier trained specifically for answer verification behave differently from the prompted VLMs used here?”; “Under a joint attack […] on several verifiers, does the combined gate fail as completely as a single readout does?” | We added two trained verifiers, a reward model and a critique-and-score evaluator, along with combinations of their scores, and found that a trained verifier can be inverted and that combining verifiers does not prevent it. A direct attack takes the reward model’s AUROC to – and the combined score to –, with the answers unchanged. |
| TwMU.9 “Randomized defenses and adaptive purification are not tested”; “JPEG purification is evaluated only against a non-adaptive attacker”; “JPEG at quality 50 removes between a quarter and two thirds of the manipulable range.” | We added adaptive attacks on JPEG purification and randomized smoothing, and both defenses fail once the attacker accounts for them. Attacked AUROC falls to – under JPEG and to under smoothing, with the answers unchanged. |
| TwMU.10 “The threat therefore applies mainly to open-weight deployments where the attacker controls the image and has gradient access”; “The paper should state this scope more prominently in the abstract and introduction.” | We added attacks on commercial API models, where GPT-5.4-nano’s AUROC falls from to using only returned token probabilities, so the vulnerability does not require open weights or gradients. We also revised the abstract, introduction, and threat model to state the scope of the core evaluation, which is white-box and per-image. |
Summary. You asked for (1) a contribution beyond extending a known attack, including why confidence is movable, (2) broader model coverage, (3) broader readout coverage, (4) stronger defenses, and (5) a clearer statement of scope. We addressed (1) with a tested hypothesis for what makes confidence movable; (2) with the Qwen2.5-VL family up to 32B; (3) with direct attacks on self-consistency, verbalized confidence, and a combined readout; (4) with joint model–probe adaptation, a second robust encoder, trained and combined verifiers, and adaptive attacks on JPEG and smoothing; and (5) with inversion through a commercial API and a revised statement of the threat model.
We thank you again for the constructive feedback and invite you to read our detailed replies to each of your ten concerns below. We believe they address your concerns and hope you will consider raising your score.
TwMU.1, in detail “The paper itself states that a confidence attack with the prediction held fixed ‘is therefore settled’”; “The attack is standard PGD, and all seven readouts are existing methods”; “The main conceptual contribution is therefore extending a known attack to open-ended VLM generation and showing that common defenses do not stop it.”
Thank you for raising this. We agree that fixed-prediction confidence attacks are established and PGD is standard. We evaluate published estimators alongside an answer-coupled readout and control constructed for this study (Section 3; Appendix A.1). The contribution lies in answering two questions: what happens when a generative system relies on confidence to decide which answers to trust, and what different safeguards actually protect.
First, consider what the gate is supposed to do. It accepts answers judged reliable and sends uncertain ones for review. We show that an attacker can turn this policy against the system without changing the answers: errors are accepted automatically, while correct answers are sent for unnecessary review. Across all twelve core cells at natural error rates, the attacked gate selects less accurate answers than having no gate (Section 4; Appendix F.1, Table 36). We then test the complete adversarial workflow: first create a wrong answer, then keep that wrong answer fixed while raising its confidence. Up to 84% of initially correct items become wrong answers that pass the gate (Section 4; Appendix F.4, Table 39). We thus demonstrate an attack that defeats both the answering model and the check intended to catch its mistake.
The vulnerability also extends beyond short answers: the tested readouts invert across all four models even when the complete answer and its explanation remain exact (Section 5; Appendix C.5, Table 14). We separately test whether the attacker needs (a) correctness labels or (b) model gradients. Without correctness labels, six of seven readouts invert throughout the core matrix (Section 4; Appendix D.2, Table 22). Our new commercial API experiment establishes inversion without gradients: correctness-aware score-only search takes GPT-5.4-nano’s AUROC from to (Section 4; Appendix I.2, Table 51).
That brings us to the second question: would a better confidence mechanism prevent this? We show that (a) training a probe to resist manipulation on frozen features sacrifices its ability to distinguish errors (Section 5; Appendix G.1, Table 41); (b) our new joint model–probe adaptation improves resistance while keeping clean confidence informative (Section 5; Appendix G.2, Table 42); and (c) an independent verifier helps against transferred attacks, but loses discrimination when directly targeted (Section 6, Table 2; Appendix H, Table 47). These results show why resistance and retained discrimination must be evaluated together, and why an independent check provides protection only under particular attacker-access assumptions.
These findings are more than a catalogue of failed defenses. We establish how confidence-based oversight, intended to make automated answers more reliable, can instead make trust decisions worse. We distinguish what different monitoring designs protect and demonstrate improvements in resistance alongside their costs. TwMU.2 presents the new hypothesis test and intervention addressing your request for greater explanatory depth.
Author Response to Reviewer TwMU (1/2)
We thank Reviewer TwMU for the detailed review and for recognizing the strict, verified answer-preservation constraint, the decomposition, and the honest reporting.
How to read our response. Each row of the table below gives one of your concerns (TwMU.1 to TwMU.10), quoted from your review, with a short, direct answer beside it. The table continues in our next message, which ends with a summary. Detailed replies then follow in this thread in the same order, titled TwMU.1, in detail to TwMU.10, in detail. They provide a more detailed response and point to the corresponding changes in the uploaded revised manuscript, where changes are marked in blue.
Our response to TwMU.6 to TwMU.10 continues in the next message.