acceptodds
Under review as a conference paper at ICLR 2027

VisualTrust: Video Reasoning under On-Screen Claim Framing

Abstract

On-screen text can describe what a video shows, or it can frame what the video is supposed to mean. Current benchmarks treat both alike, testing whether a model can read the text rather than whether it should trust it. We study claim deference, multimodal large language models' tendency to let an on-screen claim override conflicting visual evidence. We introduce **VisualTrust**, a counterfactual benchmark with 5,063 cases across 14 claim-fact classes. Each case pairs a question with three versions of the same video: one with no claim, one with an accurate claim, and one with a misleading claim that alters a single grounded fact. The option the misleading claim supports is the trap answer. Across seven open-weight and four proprietary models, mean accuracy falls from 90.1% without a claim to 51.9% with a misleading claim. Among initially correct responses, 42.8% flip to the trap answer and only 0.6% become other errors. Strong no-claim accuracy does not ensure resistance. We also introduce **DEAR** (Decoupled Evidence Arbitration and Reading), a training-free agent for video question answering that separates evidence arbitration from reading. An independent check reports conflicting evidence, a selector weighs it without a fixed modality priority and blinds what it withholds, and the reader answers from the resulting video in a fresh context. On 500 videos across three proprietary models, DEAR raises accuracy under misleading claims from 47.2–64.0% to 81.8–85.2% and reduces trap-answer selection from 34.4–50.8% to 11.6–15.0%, while preserving no-claim accuracy.

Primary area:
datasets and benchmarks
Keywords:
video understanding, multimodal large language models, on-screen text, counterfactual benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gWMvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how video models can be misled by text shown on the screen, even when the video contains evidence for the correct answer. The authors introduce a benchmark called VisualTrust to study this claim deference. It keeps the underlying video and question fixed while comparing three versions: one with an accurate claim, one with a misleading claim, and one with no claim. The results show that models can follow misleading claims, and their written explanations sometimes justify these claims despite conflicting visual evidence. So the authors also introduce DEAR, which separates deciding what evidence to use from answering the question. It checks for conflicting evidence, selectively hides content, and then lets the model answer from the remaining evidence in a fresh context, without requiring additional training.

Critical strengths

One strength is the way the paper separates understanding a video from being influenced by on-screen text. Keeping the video and question fixed, and examining cases the model originally answered correctly, makes the effect of misleading claims easier to identify.

DEAR also has practical value because it does not require retraining model, so it can be used with proprietary models. The results show that it helps models handle misleading claims while maintaining performance in the other conditions. This makes it promising for video applications where captions may be inaccurate or misleading.

Critical weaknesses

The baseline comparison could be stronger. The authors compare DEAR with direct answering and self-review, but they do not provide a comparison with a method specifically designed to handle misleading on-screen claims. A comparison with VisualTextTrap’s mitigation method on a compatible open-weight model would help show more clearly what DEAR adds beyond existing approaches, since it's the closest concurrent work. Since the two methods have different training requirements, these differences should be considered when interpreting the comparison.

One concern is that DEAR may hide useful information along with the misleading part of a claim. A caption can get one detail wrong while still providing another information needed to answer the question. Although DEAR considers the question when deciding what to hide, masking a text region could still remove both. Since VisualTrust keeps questions answerable without the added claim, the current experiments do not fully test this possibility. Examples with accurate parts of a partly misleading caption would help clarify this limitation.

I could follow the main idea, but some sections took a few readings. Several sentences pack in a lot of detail at once. Explaining the terms in simpler language and walking through the steps more gradually would make the paper easier to read. For example, I believe the comparison between DEAR and VisualTextTrap also deserve more detailed theoretical explanation with some examples. Section 2.2 lists the differences, including matched claim pairs and answer-source screening, but the explanation is brief. The baseline descriptions in Section 6.2 could also be more explicit about how the two direct prompts differ and what the self-review step asks the model to check.

Questions for authors

Do you have any existing results comparing DEAR with VisualTextTrap or any suitable targeted mitigation method on a compatible open-weight model? If so, could you share how they compare in accuracy, TFR and computational cost? This would help clarify the practical advantages of DEAR’s training-free approach.

Could you explain with more concrete theory about what your proposed method allows to measure beyond VisualTextTrap? In particular, an example showing why matched claim pairs and answer-source screening change the interpretation of the results would help me assess the contribution.

What are the main differences between the benchmark prompt and the reader prompt used for direct answering? Also, does Self-Review explicitly ask the model to check for disagreement between on-screen text and visual evidence, or does it ask the model to reconsider its answer more generally?

4 comments

Thank you for the constructive review and for noting that DEAR needs no retraining and can be used with proprietary models. Below we answer each weakness and question with new experiments and examples.

Comparison with VisualTextTrap (W1, W3, Q1, Q2).

Our work differs from VisualTextTrap in four ways. The first three concern the benchmark, matched claim pairs, answer-source screening, and native claims and timed exposure. We explain each with an example in the general comment. The fourth concerns the mitigation.

(4) Intervention only where a conflict is found. DEAR changes nothing about the model, and where the checker finds no conflict the reader answers the original video exactly as before. Its no-claim accuracy never falls below Direct's by more than a point on any of the three models. VisualTextTrap's mitigation, VTHM-MoE, trains the model for this failure, with an OCR encoder and a visual encoder that separate overlay text from the scene, four expert modules pre-trained for it and a router, on 7,000+ LLaVA-Video samples, and the trained model is what answers every video, with or without an overlay. Its weights and code are not released, so we cannot compare the two methods on the same samples.

Stronger baselines and inference cost (W1, Q1).

The instruction and critic baselines we added need no training. We ran them on Qwen3.5-Plus, Seed 2.1 Pro and Gemini-3.8-Flash over the Table 2 videos under the three conditions. We describe every method and prompt in the general comment and report the results in Table R1. Below, Direct is the version with the reader prompt, since every other method builds on that prompt. The source-aware and the visual-priority instructions both correspond to the chain-of-thought baseline in the VisualTextTrap paper, and the visual-priority instruction is the targeted mitigation that needs no training, so it is the comparison we can run.

On Qwen3.5-Plus and Seed 2.1 Pro, the visual-priority instruction closes only part of the gap between Direct and DEAR under misleading claims. On Gemini-3.8-Flash it closes most of the gap, at a cost under accurate claims, so what any training-free mitigation adds depends on how much the model defers to begin with.

In Table R2 we also compare with three systems that answer from extracted evidence, CRAFT, TRACE and MARQUIS. Under misleading claims the three do no better than Direct. Each system lists the on-screen claim among its observations as a fact about the video, so the reader sees the claim again in the evidence, and no observation marks it as disputed.

Table R3 gives the cost of every method above. Relative to Direct, the instructions cost nothing extra, the critic about twice as much and DEAR about three times. The three evidence-grounding systems are all slower than DEAR, and only TRACE uses fewer tokens.

Clarifications (W3, Q3).

The benchmark prompt is the video followed by the question and its lettered options, with no instruction. The reader prompt adds, before the question,

You are answering a question about the supplied material. Base your answer on what the material shows, and tie each observation to where it appears. Keep what you observed apart from what you infer, and say what remains uncertain. Answer the question with a brief justification.

Self-Review appends the model's own response and

You are reviewing your response to the video question. Check it against the supplied video and give your final response.

It asks for a general reconsideration and does not mention text, claims or conflicts. A variant that asks the model, while reviewing, to look for disagreements relevant to the question scored 54.4% under misleading claims on Qwen3.5-Plus, no better than Direct under the same prompt, 53.6%.

In the general comment we give the two instructions and describe the independent critic. DEAR uses the reader prompt unchanged on the video with the withheld parts removed. The revision adds these clarifications to Section 6.2, with the setup of every compared method and what Self-Review asks, and lists every prompt verbatim in Appendix G.

DEAR may hide useful information together with the misleading part of a claim (W2).

It does, and the accurate-claim column of Table R1 measures the cost. On Gemini-3.8-Flash DEAR gives up 1.6 points relative to Direct with the reader prompt under accurate claims and stays above its own no-claim accuracy, and on Qwen3.5-Plus and Seed 2.1 Pro the difference is within 0.5 points. Where DEAR loses under an accurate claim, the checker has reported a conflict that is not there, a false positive, and the selector has withheld an accurate claim the model was using. The selector does not judge whether a reported claim is accurate. It decides what counts as evidence for the question asked, and a rendered assertion is not evidence for its own content, so a claim reported as disputed is withheld whether it is accurate or misleading.

The pandas claim under (1) in the general comment is an example. "Three pandas on one swing—hold on tight!" is wrong about the count and right about the rest. DEAR withholds the whole claim, and the reader counts two pandas on the swing. Withholding a claim leaves the reader where it would be with no claim, so the most DEAR can lose is what the claim adds, and the same column measures that loss on VisualTrust.

Changes in the revised PDF. Section 2.2 explains matched claim pairs and answer-source screening with the examples in the general comment and compares with VisualTextTrap. Section 6.2 describes the setup of every method in Table 2, including what Self-Review asks, Appendix F reports the three evidence-grounding systems and the cost of every method, and Appendix G lists every prompt verbatim. Table 2 reports accurate-claim accuracy next to misleading-claim accuracy for every model, and Appendix H Limitations states the cost of withholding an accurate claim.

Reviewer_gWMvOpenReview ↗

Official Comment by Reviewer gWMv

Thank you for addressing my concerns and adding new experiments. I have no further query and would keep my score as it is.

Thank you for reading our response and giving a positive assessment. Your suggestions help improve our paper. The revised PDF includes the clarifications you asked for, with examples of matched claim pairs and answer-source screening in Section 2.2 and the setup of each baseline in Section 6.2.

Reviewer_Z4oeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates claim deference in video MLLMs under on-screen claim framing and introduces VisualTrust, a counterfactual benchmark with 5063 cases. Experiments show that models are easily misled by false on-screen claims. To address this, the authors propose DEAR, a training-free agent utilizing independent consistency checking, selective media blinding, and a fresh reader context to significantly improve robustness.

Critical strengths
  1. Novel Problem Formulation: Systematically studies and defines claim deference regarding on-screen video text, moving beyond standard OCR or object hallucination.
  2. Rigorous Benchmark: Introduces a well-controlled counterfactual benchmark VisualTrust that effectively isolates the impact of claim content.
  3. Practical Mitigation: The proposed DEAR framework is training-free and compatible with both open and proprietary models, effectively mitigating blind trust in misleading claims.
Critical weaknesses
  1. Dependency on Video Preprocessing: Different models rely on vendor-recommended preprocessing and frame-sampling strategies, which may partly confound cross-model comparisons.
  2. Computational Overhead: As an inference-time agent framework, DEAR requires extra model calls and video manipulation steps, increasing inference latency and compute costs.
Questions for authors

In complex long-form videos with multiple conflicting claims, what is the false-negative rate of the independent consistency checker, and could this become a performance bottleneck?

2 comments

Thank you for the positive assessment and for the precise question about the checker. We are encouraged that you see the benchmark as well controlled and DEAR as practical for proprietary models. Below we answer the two weaknesses and the question with new measurements, and we hope they settle the remaining concerns.

W1. Vendor-recommended preprocessing may confound cross-model comparison.

We use each model's vendor-recommended preprocessing in the main text because it is the input the model was built for, and the paper's claims do not rest on comparing models with each other. Every result is a within-model paired contrast on the same video and question, so each model's trap-flip rate is measured against its own no-claim answers under its own preprocessing. To check that the deference is not an artifact of those settings, we re-ran Direct on the full benchmark for eight models under one policy. Every model receives the same video file, sampled at 0.5 frames per second over the full duration, scaled to at most 672 pixels on the long edge and without audio. Gemini's API does not let us fix the pixel budget, so Gemini is not in this run.

ModelNo claimAccurateMisleadingTrap-flip rate ↓Trap-flip rate in Table 1
Qwen3.5-9B81.298.438.353.647.5
Qwen3.5-27B84.998.740.152.944.3
Qwen3.5-35B-A3B84.198.538.754.247.7
Qwen3.5-122B-A10B85.398.641.252.144.5
Gemma-4-26B-A4B80.597.538.852.452.7
Gemma-4-31B84.297.948.643.144.9
Qwen3.5-Plus85.597.641.052.547.5
Seed 2.1 Pro90.499.045.649.646.8

Every model shows the same deference under one policy as under its vendor's settings.

W2. Computational overhead.

DEAR makes two model calls on a sample where the checker reports no conflict and four where it does. Table R3 in the general comment gives its cost next to the two Direct baselines of Table 2, which answer from the video in one call, and the other baselines we added, on Qwen3.5-Plus over the Table 2 samples. It also shows how long OCR, masking and upload take outside the model calls. DEAR costs about three times Direct in tokens and model time.

Q. The checker's false-negative rate on long videos with several conflicts.

The checker's false-negative rate is the share of misleading-claim samples on which it does not catch the conflict, counted over the samples the model answers correctly with no claim. We measured it on the three models, with videos split into short and long at three minutes.

Video lengthQwen3.5-PlusSeed 2.1 ProGemini-3.8-Flash
up to 3 min5.42.19.4
over 3 min23.14.221.3
all7.82.411.2

When the checker misses, the claim stays in the video and the reader answers it as Direct with the reader prompt in Table 2 does, so a miss costs the gain on that sample and nothing more. On the videos over three minutes, where the rate is highest, DEAR still outperforms Direct by 12 points on Qwen3.5-Plus, 28 on Seed 2.1 Pro and 20 on Gemini-3.8-Flash. One pass can report several conflicts.

Changes in the revised PDF. Section 6.2 summarizes the inference cost, including the number of model calls DEAR makes. Appendix F gives the cost of every method and the checker's false-negative rates on short and long videos.

Dear Reviewer Z4oe,

Thank you for your constructive review. We hope our response has addressed your comments on preprocessing, cost and the checker. If anything is still unclear, we would be glad to clarify it before the discussion closes.

Best regards,

The Authors

Reviewer_GjshOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VisualTrust studies how video QA models can be misled by textual claims shown in the video, even when the visual evidence suggests otherwise. The authors introduce VisualTrust, a counterfactual benchmark where they modify a single fact in the on-screen text while keeping the original video and question unchanged. Through evaluations on 11 video models, they show that even strong models often rely on misleading textual claims rather than the actual visual evidence. Additionally, they propose DEAR, a training-free approach that uses a separate agent to identify conflicting evidence and selectively remove it before the model generates its answer. The experiments show that DEAR substantially improves accuracy under misleading claims.

Critical strengths
  1. The paper investigates an important failure mode in video question answering, where misleading on-screen textual claims can override visual evidence. The controlled counterfactual setup is particularly useful for isolating this behavior by modifying textual claims while keeping the underlying video and question unchanged.

  2. The evaluation across 11 video MLLMs, including open-weight and proprietary models, exposes significant weaknesses even in strong-performing models. The analysis of trap-answer flips and reasoning traces provides useful insights into how models rely on misleading textual information.

  3. DEAR provides a simple inference-time strategy for separating evidence arbitration from answer generation. The reported improvements are substantial, and the ablations on evidence blinding and reader isolation provide some support for the proposed design.

Critical weaknesses
  1. It is unclear to what extent VisualTrust constitutes a new video QA benchmark rather than a controlled adversarial robustness benchmark based on counterfactual textual claims. Related benchmarks, particularly VisualTextTrap, already investigate misleading text overlays and their effects on visual reasoning. Although the authors discuss these differences, the additional contribution of VisualTrust, beyond its controlled single-fact edits and taxonomy, seems narrow.
  2. Existing approaches such as CRAFT ACL 2026, TRACE ACL 2026, and MARQUIS ACL 2026 investigate evidence-grounded video reasoning, structured evidence extraction, and claim verification. Although these methods address somewhat different tasks, stronger comparisons with applicable evidence-grounding or critic-based approaches would help establish the distinct contribution of DEAR.
  3. Most importantly, the benchmark assumes that the recorded visual evidence determines the correct answer when it conflicts with an added textual claim. However, the models are not explicitly informed which source should be prioritized, and on-screen text may itself be legitimate evidence depending on the question. Although the authors screen questions to reduce this ambiguity, it remains unclear how much of the observed failure could be addressed through better source-aware prompting. A simple baseline explicitly instructing models to prioritize recorded visual evidence over added claims, when appropriate, would help determine whether DEAR is necessary.
  4. DEAR requires multiple model calls and additional OCR-based evidence processing and media manipulation. Although the paper demonstrates accuracy improvements, a comparison of inference latency and computational cost would appropriate the advantage.
Questions for authors
  1. Could you please elaborate in what ways does is this benchmark different than the VisualTextTrap beyond the control and what is it's unique advantage?
  2. Current evaluation of DEAR seems to be against models that do not necessarily explicitly ground their claims in their evidence or operate on structured evidence or memories and have critics (I have pointed few in the weakness), it would be better that important baselines to be identified and compared against. The current method DEAR seems to be too "naive" to be considered a solid methodological contribution.
  3. Can you add baseline where models are advised to assume visual evidence as a true evidence, what will the performance of these baselines be?
  4. Why should we not call this simply a counterfactual attach benchmark?
  5. Please evaluate inference costs and performance tradeoff with all the baselines to keep the gains in perspective.
13 comments

Thank you for the detailed review. We are glad that you consider the failure mode we study important and the improvements from DEAR substantial. Below we respond to each point with new baselines and analyses.

W1 / Q1 / Q4. What VisualTrust measures, how it differs from VisualTextTrap, and why we do not call it an attack benchmark.

In the general comment we compare VisualTrust with VisualTextTrap on three choices about the benchmark, matched claim pairs, answer-source screening, and native claims and timed exposure.

We designed VisualTrust around controlled claim interventions, so "controlled counterfactual" describes how the benchmark is built. We do not call it an attack benchmark, because we do not score resistance alone. An attack benchmark asks whether injected text can move a model, and a model that ignores the text scores best. We ask how much weight a model gives the claim channel, and a model that ignores the channel loses under accurate claims. Half of the claims are accurate, and an accurate claim raises accuracy by eight points on average across the twelve models. The visual-priority baseline in Table R1 shows the loss on Gemini-3.8-Flash, where the instruction pushes accuracy under accurate claims below no-claim accuracy. What the benchmark measures is how much weight the channel deserves, with its value and its cost side by side.

W2 / Q2. Evidence-grounding and critic-based baselines, and "too naive".

DEAR needs no training and no access to weights, makes two to four model calls, and runs on any model behind an API. It does not fix a modality priority. No channel is trusted or distrusted by default. When the checker finds a conflict about the asked fact, the selector withholds the text that states the disputed answer, on screen or in speech, and keeps the footage that shows the fact, because a statement is not evidence for what it states. A fresh model then answers from what remains. To show what this is worth, we compare DEAR with the instruction and critic baselines described in the general comment. Table R1 reports the results on Qwen3.5-Plus, Seed 2.1 Pro and Gemini-3.8-Flash.

Direct below means Direct with the reader prompt, which every other method builds on. Under misleading claims DEAR is the most accurate method on all three models and selects the trap answer least often. The visual-priority instruction helps least where the model defers most. On Qwen3.5-Plus it lifts misleading-claim accuracy from 53.6 to 60.2, where DEAR reaches 85.2, and Seed 2.1 Pro shows the same picture. On Gemini-3.8-Flash the instruction comes within 2.6 points of DEAR, but under accurate claims it pushes the model below its own no-claim accuracy, because a fixed priority discounts an accurate claim as readily as a misleading one. The critic does about as well as the instruction on Qwen3.5-Plus and better on Seed 2.1 Pro. On Gemini-3.8-Flash it is worse than Direct, because the model, shown the disagreement, often names both options instead of choosing one, and an answer that names two options counts as incorrect. Removing the claim from the video before the model answers is the one change that removes the deference on all three models and keeps accuracy under accurate claims at or above no-claim accuracy.

One sample on Qwen3.5-Plus shows why the instruction and the report fall short where DEAR does not. The question is "How many people can be seen holding cameras in the video?", the scene shows two, and the misleading overlay says three. Direct answers three, and its trace reads, "presumably a third reporter is also present holding one as indicated by the text". The model treats the rendered claim as visual evidence, so telling it to prioritize the scene changes nothing, and with the visual-priority instruction the trace still reads, "the text confirms the total count of people holding cameras is three". With the critic, the model receives the report, which pairs the two accounts and states that they disagree, and answers three anyway, reading, "such text overlays are intended to define the scene's content". A model that notices the conflict reconciles it in favor of the claim, so a report of the conflict is reconciled away in the same way. DEAR, with the overlay withheld, answers two. These are the behaviors the traces in Section 5.4 document, and DEAR's choices answer them. The claim is removed from the media rather than argued against, and the model never sees the report.

DEAR and the independent critic differ in two choices, whether the disputed content is removed from the video and whether the reader sees the report. To separate the two, we ran all four combinations on Qwen3.5-Plus with the same videos and reader prompt.

Misleading-claim accuracy / trap-answer rateReader does not see the reportReader sees the report
Original video53.6 / 43.2 (Direct)61.2 / 35.4 (independent critic)
Blinded video85.2 / 11.6 (DEAR)75.6 / 19.8 (blinded video + report)

The gain comes from removing the disputed content, with or without the report. The report helps only while the claim is still in the video. Once the claim is gone it costs the reader 9.6 points, because it brings the disputed claim back into view as text. The reader therefore never sees it.

We agree that methods with an evidence mechanism are relevant baselines for DEAR. Following your suggestion, we compare with CRAFT, TRACE and MARQUIS in Table R2.

Without a claim and under accurate claims the three stay within three points of Direct, so the adaptation costs them little. Under misleading claims they do no better than Direct and select the trap answer as often or more. Their observations record the on-screen claim as a fact next to what the scene shows, so the reader receives the claim twice, once in the video and once in the evidence, and nothing that marks it as disputed. Extracting evidence does not help here. What helps is noticing that two accounts disagree and keeping the disputed one away from the reader, and the independent critic above, which reports the disagreement instead of a list of facts, already does better than all three.

W3 / Q3. A baseline told to prioritize recorded visual evidence.

We added the source-aware and the visual-priority instructions shown in the general comment, each appended to the reader prompt and identical in all three conditions. Under misleading claims the visual-priority instruction recovers only a small part of the gap between Direct and DEAR on Qwen3.5-Plus and Seed 2.1 Pro. On Gemini-3.8-Flash it comes within 2.6 points of DEAR but falls below no-claim accuracy under accurate claims. The source-aware instruction, which leaves the choice of source to the model, helps only on Gemini-3.8-Flash and hurts on the other two. Table R1 gives every number.

Telling the model which source to trust does not stop it from explaining the conflict away. On the 324 misleading-claim samples on Qwen3.5-Plus where the critic's report pairs an account from the on-screen text with one from the scene, the reader with the report still selects the trap answer 40.1% of the time, against 58.3% for Direct on the same samples. Told instead which source to prioritize, it does so 42.6% of the time. With the claim removed from the video, 9.6%. A model that can see the claim finds a reading under which the claim and the scene agree, as the camera-count traces under W2 show, whether it is told that the sources disagree or which one to prefer. Removing the claim leaves it nothing to reconcile.

The model does this even when the question says where to look. In the revolving-door case of Appendix E the question asks which English word is printed on the glass panels of the revolving door, and the misleading overlay says "Welcome". The trace reads "I clearly see 'Sandhill' on the glass. But the text overlay says 'Welcome'", then "the question is clearly keyed to the text overlay", and answers Welcome. The question says where to look, the model reads the correct word there, and it still changes what the question is about so that the overlay fits.

W4 / Q5. Inference cost against every baseline.

On a sample where the checker reports no conflict, DEAR makes two model calls, the checker and the reader. Where it reports one, the selector runs twice before the reader, four calls in all. Table R3 gives the cost of every method in Tables R1 and R2, and the time OCR, masking and upload add outside the model calls. DEAR costs about three times Direct, the critic about twice and the instructions nothing extra. The evidence-grounding systems all take longer than DEAR, and two of them use more tokens.

Changes in the revised PDF. Section 2.2 compares VisualTrust and VisualTextTrap on the three design choices we describe in the general comment. We extend Table 2 with the instruction and critic baselines, with Gemini-3.8-Flash in place of Gemini-3.5-Flash, and replace Table 10 with the ablation under W2. Section 6.2 describes the setup of every method in Table 2, and summarizes the inference cost. Appendix F reports CRAFT, TRACE and MARQUIS and the cost of every method, and Appendix G lists every prompt verbatim.

We welcome further questions.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General comment

We thank the reviewers for their careful and specific feedback. We agree that DEAR should be compared with stronger baselines, and we have added the following experiments and analyses. We report what more than one reviewer asked about in this comment, and the rest in each reviewer's thread.

  • Baselines for DEAR on three models, a source-aware and a visual-priority instruction, an independent critic and three evidence-grounding systems, CRAFT, TRACE and MARQUIS (Reviewers Gjsh and gWMv).
  • An ablation that separates DEAR's two design choices (Reviewer Gjsh).
  • Inference cost of every method (Reviewers Gjsh, Z4oe and gWMv).
  • The checker's false-negative rate on short and long videos (Reviewer Z4oe).
  • Direct re-run on the full benchmark for eight models under one preprocessing policy (Reviewer Z4oe).
  • A screening experiment on the benchmark's own questions (Reviewers Gjsh and gWMv).

Since Google deprecated Gemini-3.5-Flash on October 8, 2026, we replace it with Gemini-3.8-Flash and re-run every method on it. The abstract and Section 1 report new numbers for this reason: Table 1 adds Gemini-3.8-Flash as a twelfth model, and Table 2 reports it in place of Gemini-3.5-Flash.

In the revised PDF, we include Gemini-3.8-Flash in Table 1, extend Table 2 with the instruction and critic baselines, and replace Table 10 with the ablation in the response to Reviewer Gjsh. In Appendix F we report the evidence-grounding systems, the cost of every method and the checker's false-negative rates. We list every prompt verbatim in Appendix G and state in Appendix H Limitations what withholding an accurate claim costs. Text that is new or changed is shown in blue.

Comparison with VisualTextTrap (Reviewers Gjsh and gWMv).

We propose VisualTrust to measure claim deference, that is, how far a misleading on-screen claim moves a model's answer away from what the video shows. The concurrent work VisualTextTrap also compares videos with and without an overlay. VisualTrust differs from it in three choices about the benchmark.

(1) Matched claim pairs. Every case carries the same question and options under three conditions, no claim, an accurate claim and a misleading claim. The misleading claim is the accurate one with a single fact changed, rendered with the same typography, placement and timing. For the question "How many pandas are on the swing?", the pair is

Accurate claim:   Two pandas on one swing—hold on tight!
Misleading claim: Three pandas on one swing—hold on tight!

The two claims differ in only one word, so wording, length and salience cannot explain a difference between the two claim conditions. What remains is claim content, and it moves answers both ways. Accuracy rises with the accurate claim and falls with the misleading one, and nearly every new error lands on the asserted option. In VisualTextTrap most video–question pairs appear under only one condition, and only 389 appear under all three. Its congruent and contradictory texts are also generated separately, so a drop there can come from wording or salience as well as from the claim.

(2) Answer-source screening. For every question we checked whether the on-screen claim itself could count as a legitimate answer. When it could, we rewrote the question so that only the scene can answer it. "What is the price?" can be answered by a claim that states a price, so it became "What is the price on the shelf tag?" Without this step, a flip can be a legitimate answer. Asked the original question, one model wrote in its reasoning, "The video contains a text overlay that explicitly answers this question." With this step, a flip is deference. VisualTextTrap has no such step. To measure the difference, we evaluated Qwen3.5-Plus on the 762 questions we rewrote, once with the original question and once with the rewritten one. No-claim accuracy is the same in both cases, and under the misleading claim the trap-flip rate falls from 37.1% with the original question to 22.4% with the rewritten one. The flips that disappear were legitimate answers to an open question, and the flips that persist are what the benchmark measures.

(3) Native claims and timed exposure. Part of the benchmark uses creators' own claims from BOVText, with their own wording and their own timing, and our claim pairs are single-fact edits of them. In VisualTextTrap every text is generated and overlaid on every frame. The channel we measure is one that real videos already carry, not one constructed for the benchmark, and that is one reason we do not call the setup an attack.

VisualTextTrap is a concurrent, independently developed preprint, and as of November 2026 its project page has no downloadable benchmark, code or model weights, so we could not compare on its data or with its mitigation, VTHM-MoE.

Compared methods (Reviewers Gjsh and gWMv).

To show what DEAR is worth, we added the following baselines on the Table 2 videos under the three conditions, on Qwen3.5-Plus, Seed 2.1 Pro and Gemini-3.8-Flash. For example, the benchmark prompt is the video followed by

Afterward, who puts the brown hat on whose head?

A. The woman in black puts it on the child's head.
B. The child puts it on their own head.
C. The child puts it on the uniformed ranger's head.
D. The uniformed ranger puts it on the child's head.

Each method's input:

  • Direct (benchmark prompt). {video} {question} {options}, as above.
  • Direct (reader prompt). {video} {reader prompt} {question} {options}. The reader prompt is
You are answering a question about the supplied material. Base your answer on what the material shows, and tie each observation to where it appears. Keep what you observed apart from what you infer, and say what remains uncertain. Answer the question with a brief justification.
  • Source-aware. {video} {reader prompt + source-aware instruction} {question} {options}. The instruction is
The material can carry information from more than one source. Decide which source answers the question, and rely on it.
  • Visual-priority. {video} {reader prompt + visual-priority instruction} {question} {options}. The instruction is
For questions about events or properties visible in the recorded scene, prioritize the recorded visual evidence when an added claim conflicts with it. When the question asks what a text says or displays, use that target text.
  • Independent critic. {video} {reader prompt} {question} {options} {report}. A critic in a separate call watches the original video with the question stem alone and looks for places where the video contradicts itself about the asked fact. Its report pairs the two accounts of each disagreement, with their source and time, and states what they disagree on. It can hold several disagreements or none, and it never says which account is right. The model then answers from the original video with the report attached. The critic is DEAR's checker, so the two methods differ only in what the model sees. For the question above, under the misleading claim the critic reported one disagreement:
Fact in dispute: who puts the brown hat on the boy's head
Account A (scene, 0:04): the boy picks up the brown hat and places it on his own head.
Account B (on-screen text, 0:06–0:11): "The park ranger places the brown hat on our junior ranger's head—now for the pledge."
Disagreement: whether the boy or the park ranger placed the hat on the boy's head.
  • DEAR. {blinded video} {reader prompt} {question} {options}, where the blinded video is the original with the withheld parts removed.

Table R1. DEAR and the added baselines on three models.

MethodNo claimAccurateMisleadingTrap-answer rate ↓
Qwen3.5-Plus
Direct (benchmark prompt)89.496.649.447.6
Direct (reader prompt)91.098.253.643.2
+ source-aware instruction90.497.443.254.0
+ visual-priority instruction91.096.860.236.8
Independent critic88.296.461.235.4
DEAR90.498.085.211.6
Seed 2.1 Pro
Direct (benchmark prompt)91.098.047.250.8
Direct (reader prompt)92.298.450.247.4
+ source-aware instruction92.098.048.449.8
+ visual-priority instruction92.297.654.044.4
Independent critic89.296.859.039.6
DEAR94.498.082.415.0
Gemini-3.8-Flash
Direct (benchmark prompt)93.497.674.824.2
Direct (reader prompt)93.897.678.220.2
+ source-aware instruction93.095.882.416.4
+ visual-priority instruction94.090.489.09.0
Independent critic93.888.076.617.6
DEAR94.096.091.67.4

For VisualTrust, the relevant baselines for DEAR also include methods with an evidence mechanism. We compare with three of them, CRAFT, TRACE and MARQUIS. All three answer a query by first extracting observations from the video, one sentence per thing it shows, says or displays, each tagged with its source and time, and then writing the answer from those observations. CRAFT draws its observations from transcribed speech and selected keyframes and verifies them with an entailment-based critic loop. TRACE builds a timeline of on-screen text and detected objects, grounds the query in it to pick frames, and extracts observations from those frames. MARQUIS extracts observations conditioned on the query and ranks them by a calibrated confidence. They were built for collections of videos, so with one video we skip retrieval and cross-video consolidation and run each system's evidence stages unchanged. The reader then answers from the original video and the system's observations, and every model call uses Qwen3.5-Plus.

Table R2. Evidence-grounding systems on Qwen3.5-Plus.

MethodNo claimAccurateMisleadingTrap-answer rate ↓
Direct (reader prompt)91.098.253.643.2
CRAFT88.695.643.451.8
TRACE89.095.845.850.4
MARQUIS89.096.848.448.0
DEAR90.498.085.211.6

Inference cost (Reviewers Gjsh, Z4oe and gWMv).

DEAR runs the checker and the reader on every sample, and the selector twice when the checker reports a conflict. The table gives the cost of every method in Tables R1 and R2 on Qwen3.5-Plus over the Table 2 samples.

Table R3. Inference cost on Qwen3.5-Plus. Tokens are the mean per sample over all of a method's calls, model time is the median per sample of the time those calls take, and the last two columns are relative to Direct with the reader prompt.

MethodModel callsTokens per sampleModel time per sampleTokens vs DirectTime vs Direct
Direct (benchmark prompt)145.6k26.6 s1.0×1.0×
Direct (reader prompt)146.7k25.8 s1.0×1.0×
+ source-aware instruction146.2k24.3 s1.0×0.9×
+ visual-priority instruction146.0k24.3 s1.0×0.9×
Independent critic288.4k48.7 s1.9×1.9×
DEAR3.6159.6k74.9 s3.4×2.9×
CRAFT19.4217.0k390.9 s4.6×15.2×
TRACE8.8102.9k337.3 s2.2×13.1×
MARQUIS6.5274.8k177.0 s5.9×6.9×

Outside the model calls, OCR, masking and upload add a median of 0.1 s on a sample where the checker reports no conflict and 3.6 s on one where it reports a conflict, against 74.9 s of model time.

AuthorsOpenReview ↗

Summary of the discussion

Dear Reviewers and AC,

Thank you for your time and for the discussion. With half a day left in the discussion period, we summarize our responses to the main concerns.

1. Relation to VisualTextTrap (Gjsh, gWMv)

VisualTrust and VisualTextTrap differ in motivation, findings and mitigation.

  • VisualTrust was developed independently, from a different motivation. Reading an on-screen claim is not the same as trusting it, so VisualTrust studies claim deference, a model's tendency to let an on-screen claim override visual evidence. To separate it from ordinary visual errors, each case is answered with no claim, with an accurate claim, and with a misleading claim that differs from the accurate one in a single fact.
  • On VisualTrust, (1) models follow a misleading claim even when the question specifies that the answer comes from the scene, (2) models that perform equally well on video QA can differ widely in how often they follow a misleading claim, and (3) even when a model notices the conflict, it tends to invent an explanation that reconciles the claim with the scene, as 86% of the flipped traces that acknowledge the conflict do.
  • Building on these findings, DEAR lets the model itself decide which evidence to keep and then answer in an isolated context. It works at inference time, which is different from the training-time mitigation of VisualTextTrap.

2. Baselines and ablations (Gjsh, gWMv)

  • DEAR outperforms every baseline under misleading claims. The baselines include the simpler visual-evidence instruction Gjsh suggested, and three evidence-grounding systems.
  • Ablations show that DEAR needs both a selector without a fixed modality priority and an isolated reader context. Replacing the selector with direct masking also removes the scene text the question needs, and a reader that is told about the conflict while the claim stays in the video often still follows the claim.

3. Cost and checker reliability (Gjsh, Z4oe, gWMv)

  • DEAR raises misleading-claim accuracy from 47.2–74.8% to 82.4–91.6% at about 3 times the model time of Direct. The three evidence-grounding baselines take longer, 7 to 15 times Direct.
  • A conflict the checker misses leaves the video unchanged, so the reader answers as Direct does. Even on videos over three minutes, where misses are most common, DEAR exceeds Direct by 12–28 points.

4. Video preprocessing (Z4oe)

  • Rerun under one shared preprocessing policy, all eight models show the same deference.

We hope this summary makes the discussion easier to follow.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.