- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how video models can be misled by text shown on the screen, even when the video contains evidence for the correct answer. The authors introduce a benchmark called VisualTrust to study this claim deference. It keeps the underlying video and question fixed while comparing three versions: one with an accurate claim, one with a misleading claim, and one with no claim. The results show that models can follow misleading claims, and their written explanations sometimes justify these claims despite conflicting visual evidence. So the authors also introduce DEAR, which separates deciding what evidence to use from answering the question. It checks for conflicting evidence, selectively hides content, and then lets the model answer from the remaining evidence in a fresh context, without requiring additional training.
Critical strengths
One strength is the way the paper separates understanding a video from being influenced by on-screen text. Keeping the video and question fixed, and examining cases the model originally answered correctly, makes the effect of misleading claims easier to identify.
DEAR also has practical value because it does not require retraining model, so it can be used with proprietary models. The results show that it helps models handle misleading claims while maintaining performance in the other conditions. This makes it promising for video applications where captions may be inaccurate or misleading.
Critical weaknesses
The baseline comparison could be stronger. The authors compare DEAR with direct answering and self-review, but they do not provide a comparison with a method specifically designed to handle misleading on-screen claims. A comparison with VisualTextTrap’s mitigation method on a compatible open-weight model would help show more clearly what DEAR adds beyond existing approaches, since it's the closest concurrent work. Since the two methods have different training requirements, these differences should be considered when interpreting the comparison.
One concern is that DEAR may hide useful information along with the misleading part of a claim. A caption can get one detail wrong while still providing another information needed to answer the question. Although DEAR considers the question when deciding what to hide, masking a text region could still remove both. Since VisualTrust keeps questions answerable without the added claim, the current experiments do not fully test this possibility. Examples with accurate parts of a partly misleading caption would help clarify this limitation.
I could follow the main idea, but some sections took a few readings. Several sentences pack in a lot of detail at once. Explaining the terms in simpler language and walking through the steps more gradually would make the paper easier to read. For example, I believe the comparison between DEAR and VisualTextTrap also deserve more detailed theoretical explanation with some examples. Section 2.2 lists the differences, including matched claim pairs and answer-source screening, but the explanation is brief. The baseline descriptions in Section 6.2 could also be more explicit about how the two direct prompts differ and what the self-review step asks the model to check.
Questions for authors
Do you have any existing results comparing DEAR with VisualTextTrap or any suitable targeted mitigation method on a compatible open-weight model? If so, could you share how they compare in accuracy, TFR and computational cost? This would help clarify the practical advantages of DEAR’s training-free approach.
Could you explain with more concrete theory about what your proposed method allows to measure beyond VisualTextTrap? In particular, an example showing why matched claim pairs and answer-source screening change the interpretation of the results would help me assess the contribution.
What are the main differences between the benchmark prompt and the reader prompt used for direct answering? Also, does Self-Review explicitly ask the model to check for disagreement between on-screen text and visual evidence, or does it ask the model to reconsider its answer more generally?
4 comments
DEAR may hide useful information together with the misleading part of a claim (W2).
It does, and the accurate-claim column of Table R1 measures the cost. On Gemini-3.8-Flash DEAR gives up 1.6 points relative to Direct with the reader prompt under accurate claims and stays above its own no-claim accuracy, and on Qwen3.5-Plus and Seed 2.1 Pro the difference is within 0.5 points. Where DEAR loses under an accurate claim, the checker has reported a conflict that is not there, a false positive, and the selector has withheld an accurate claim the model was using. The selector does not judge whether a reported claim is accurate. It decides what counts as evidence for the question asked, and a rendered assertion is not evidence for its own content, so a claim reported as disputed is withheld whether it is accurate or misleading.
The pandas claim under (1) in the general comment is an example. "Three pandas on one swing—hold on tight!" is wrong about the count and right about the rest. DEAR withholds the whole claim, and the reader counts two pandas on the swing. Withholding a claim leaves the reader where it would be with no claim, so the most DEAR can lose is what the claim adds, and the same column measures that loss on VisualTrust.
Changes in the revised PDF. Section 2.2 explains matched claim pairs and answer-source screening with the examples in the general comment and compares with VisualTextTrap. Section 6.2 describes the setup of every method in Table 2, including what Self-Review asks, Appendix F reports the three evidence-grounding systems and the cost of every method, and Appendix G lists every prompt verbatim. Table 2 reports accurate-claim accuracy next to misleading-claim accuracy for every model, and Appendix H Limitations states the cost of withholding an accurate claim.
Official Comment by Reviewer gWMv
Thank you for addressing my concerns and adding new experiments. I have no further query and would keep my score as it is.
Thank you for reading our response and giving a positive assessment. Your suggestions help improve our paper. The revised PDF includes the clarifications you asked for, with examples of matched claim pairs and answer-source screening in Section 2.2 and the setup of each baseline in Section 6.2.
Thank you for the constructive review and for noting that DEAR needs no retraining and can be used with proprietary models. Below we answer each weakness and question with new experiments and examples.
Comparison with VisualTextTrap (W1, W3, Q1, Q2).
Our work differs from VisualTextTrap in four ways. The first three concern the benchmark, matched claim pairs, answer-source screening, and native claims and timed exposure. We explain each with an example in the general comment. The fourth concerns the mitigation.
(4) Intervention only where a conflict is found. DEAR changes nothing about the model, and where the checker finds no conflict the reader answers the original video exactly as before. Its no-claim accuracy never falls below Direct's by more than a point on any of the three models. VisualTextTrap's mitigation, VTHM-MoE, trains the model for this failure, with an OCR encoder and a visual encoder that separate overlay text from the scene, four expert modules pre-trained for it and a router, on 7,000+ LLaVA-Video samples, and the trained model is what answers every video, with or without an overlay. Its weights and code are not released, so we cannot compare the two methods on the same samples.
Stronger baselines and inference cost (W1, Q1).
The instruction and critic baselines we added need no training. We ran them on Qwen3.5-Plus, Seed 2.1 Pro and Gemini-3.8-Flash over the Table 2 videos under the three conditions. We describe every method and prompt in the general comment and report the results in Table R1. Below, Direct is the version with the reader prompt, since every other method builds on that prompt. The source-aware and the visual-priority instructions both correspond to the chain-of-thought baseline in the VisualTextTrap paper, and the visual-priority instruction is the targeted mitigation that needs no training, so it is the comparison we can run.
On Qwen3.5-Plus and Seed 2.1 Pro, the visual-priority instruction closes only part of the gap between Direct and DEAR under misleading claims. On Gemini-3.8-Flash it closes most of the gap, at a cost under accurate claims, so what any training-free mitigation adds depends on how much the model defers to begin with.
In Table R2 we also compare with three systems that answer from extracted evidence, CRAFT, TRACE and MARQUIS. Under misleading claims the three do no better than Direct. Each system lists the on-screen claim among its observations as a fact about the video, so the reader sees the claim again in the evidence, and no observation marks it as disputed.
Table R3 gives the cost of every method above. Relative to Direct, the instructions cost nothing extra, the critic about twice as much and DEAR about three times. The three evidence-grounding systems are all slower than DEAR, and only TRACE uses fewer tokens.
Clarifications (W3, Q3).
The benchmark prompt is the video followed by the question and its lettered options, with no instruction. The reader prompt adds, before the question,
Self-Review appends the model's own response and
It asks for a general reconsideration and does not mention text, claims or conflicts. A variant that asks the model, while reviewing, to look for disagreements relevant to the question scored 54.4% under misleading claims on Qwen3.5-Plus, no better than Direct under the same prompt, 53.6%.
In the general comment we give the two instructions and describe the independent critic. DEAR uses the reader prompt unchanged on the video with the withheld parts removed. The revision adds these clarifications to Section 6.2, with the setup of every compared method and what Self-Review asks, and lists every prompt verbatim in Appendix G.