acceptodds
Under review as a conference paper at ICLR 2027

EviScene: Beyond “Is It AI?” for Agentic Image Factuality Assessment

Abstract

Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert-reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine-tuning on reviewed teacher trajectories, followed by privileged on-policy self-distillation applied to verified decisions from the policy's own investigations. On the 1,527-image test set, the trained Qwen3.5-9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same-backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Image Understanding, Multimodal Learning, Information Retrieval

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2KqQOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper points out that whether an image is AI-generated does not necessarily indicate whether the situation it depicts is true or false. To address this, the authors introduce the task of "Image Factuality Assessment." In this task, a system is provided only with an image and a general verification prompt, without any accompanying textual claims, and must determine whether the depicted situation is "substantiated" or "refuted," while providing an explanation based on external evidence. For this task, the authors developed a benchmark called EviLens and proposed a multimodal agent named EviScene. EviScene operates by iteratively performing visual inspection, external information retrieval, and evidence comparison, progressively refining the questions under investigation. The authors trained the Qwen3.5-9B model using a two-stage process, enabling it to achieve a balanced accuracy and a strict evidence-sufficiency rate.

Critical strengths

The authors collected and manually reviewed a large volume of data for EviLens. The authors conducted extensive testing of EviLens using a variety of both open-source and closed-source models.

Critical weaknesses
  1. Although the authors have collected and organized a large dataset, the distribution is quite imbalanced (3,114 substantiated images versus 7,060 refuted images). I believe the dataset could be smaller, provided that a better balance is maintained.
  2. The authors do not compare how different agent architectures affect task completion.
  3. Some experimental results in the paper lack analysis, making them difficult to interpret.
Questions for authors
  1. Could the authors clarify the distinction between the "Image Factuality Assessment" proposed in this paper and general multimodal fact-checking tasks?
  2. Could the authors clarify the final conclusions drawn in Sections 5.4 and 5.5? Currently, these sections merely present experimental results without providing sufficient analysis.
  3. Table 1 shows a significant disparity in SESR scores between models that do not employ an agent and those that do, while the differences across other experimental metrics remain relatively small. Could the authors provide a reasonable explanation for this anomaly?
3 comments

We thank the reviewer for the comments and address each point below. We have uploaded a revised PDF with changes in blue; all references below are to it.


W1: Label imbalance

The imbalance does not affect our results. BAcc and macro F1 weight both classes equally, so labeling every image refuted yields 50% BAcc. The trained policy recalls 80.11% of supported and 78.00% of refuted images (Table 1). On web-collected images, which are close to balanced with 144 supported and 145 refuted test images (Table 8), it reaches 77.21% BAcc, compared with 76.75% on the imbalanced constructed images (Table 2). The imbalance comes from controlled construction, which builds 502 to 524 refuted images for each of twelve error types so that each type is well represented (Table 9).


W2: Comparison of agent architectures

We add three variants on the full test set to Section 5.3 and Table 4a, all using Gemini 3.7 Flash with the same tools and 24-action budget. With a fixed question, the agent states one verification question from the image before any tool call and keeps it throughout. With a fixed sequence, every image goes through the same nine actions: scene perception, OCR, reverse image search, three text searches, and three page visits; the agent only fills in the tool arguments.

QuestionAction sequenceBAccSESRAverage actions per image
Adaptive, EviSceneAdaptive81.5742.897.18
FixedAdaptive77.0138.5714.02
AdaptiveFixed78.3239.699.00
FixedFixed75.8336.809.00

EviScene, which adapts both, reaches the highest BAcc and SESR with the fewest actions. Fixing the question, the sequence, or both lowers BAcc by 4.56, 3.25, and 5.74 points and SESR by 4.32, 3.20, and 6.09 points; fixing the question also nearly doubles the actions per image. A question fixed before any evidence commits the investigation to one detail, whereas EviScene revises what it checks as findings accumulate.


W3 and Q2: Conclusions of Sections 5.4 and 5.5

We rewrote both sections to state their conclusions; the revised text is in the PDF and the next comment. Section 5.4 concludes that the gains from training extend to web-collected images and to generators unseen in training. Section 5.5 concludes that prior knowledge and visual plausibility cannot substitute for evidence about the depicted situation.


Q1: Difference from multimodal fact-checking

Multimodal fact-checking depends on a given claim, which states what is to be verified. In MOCHEG (Yao et al., 2023), the claim is text and images appear among the retrieved evidence. In AVerImaTeC (Cao et al., 2025), RW-Post (Xu et al., 2026), and VeriTaS (Rothermel et al., 2026), a textual claim is paired with images or with the social-media post that carried it. When a claim is only implicit in an image, the standard pipeline first extracts it in a separate claim-extraction stage, so verification still starts from a given claim. Image Factuality Assessment supplies no claim and has no extraction stage: the system receives one image and a generic instruction, decides during investigation which depicted facts to check, and judges the depicted situation as a whole.

Multimodal fact-checkingImage Factuality Assessment
What is verifiedThe assertion stated in the claimThe situation depicted in the image
What guides the investigationThe given claim states what to look forThe system decides from visual content what to look for
Evidence alignmentEvidence must match the stated assertionEvidence must match the depicted event, time, and stage, which no text specifies

Table 5 in Appendix A.1 further compares EviLens with benchmarks for related tasks.


Q3: The SESR gap between direct judgment and agents

This gap is expected, and Section 5.2 now explains it. BAcc checks only the label, whereas SESR also requires the response to establish the decisive fact with evidence. Without investigation, a model has no such evidence, and its labels can follow a default preference: under direct judgment, Gemini 3.1 Pro recalls 96.52% of refuted but only 28.65% of supported images, while GLM-4.6V shows the reverse (Table 1). Such a label can therefore match the ground truth while its grounds do not establish the decisive fact, which keeps SESR near zero. Within investigation, SESR likewise follows how much evidence the agent can obtain: raising the action limit of the full policy from 8 to 16 adds 12.05 SESR points but only 0.79 BAcc points (Table 4b), and removing web search lowers SESR by 16.37 points but BAcc by only 4.68 (Table 3). The gap therefore measures the evidence that direct judgment cannot obtain.

Revised Sections 5.4 and 5.5

5.4 Source and Generator Analysis. The gains from training extend to web-collected images and to generators unseen in training (Table 2). The full policy performs similarly on constructed and web-collected images, with 76.75% versus 77.21% BAcc and 52.75% versus 50.17% SESR. Training improves both sources: relative to the base agent, Full gains 8.88 BAcc and 8.81 SESR points on constructed images, and 3.87 BAcc and 11.76 SESR points on web-collected images. Across generators, Full exceeds the base agent on images from all three, including GPT Image 2 and Seedream 5.0 Lite, which are not used to construct training images.

5.5 Qualitative Analysis. Prior knowledge and visual plausibility cannot substitute for evidence about the depicted situation. Figure 5 shows two complementary failure patterns of direct judgment by GPT-5.5. The first pattern treats missing knowledge as evidence of falsity. In Figure 5a, the display label names a DisplayHDR True Black 1400 certification, and GPT-5.5 rejects it because it does not know this tier; EviScene finds that VESA introduced the tier in July 2026. In Figure 5d, the image shows the launch of Plan Cabaret at the Moulin Rouge, and GPT-5.5 calls the program invented; EviScene finds the Ministry of Culture record of the launch on January 21, 2025. The second pattern accepts a plausible scene without checking the decisive detail. In Figure 5b, the Cassini poster names the correct exhibition, institution, and venue, so GPT-5.5 accepts it; EviScene retrieves the official schedule, which runs from October 14, 2026 to February 1, 2027, whereas the poster shows 2025 to 2026. In Figure 5c, GPT-5.5 accepts the depicted opening ceremony; EviScene finds official records placing berlin modern in its construction stage as of the evaluation reference date of September 1, 2026. In all four cases, GPT-5.5 knows the current date and the entities involved but lacks the specific record that decides the verdict.

Dear Reviewer 2KqQ,

Thank you again for raising the questions of agent architecture and label imbalance. In our response, we compared EviScene with variants that fix the verification question, the action sequence, or both; fixing them lowers BAcc by 3.25 to 5.74 points and SESR by 3.20 to 6.09 points (Table 4a). The imbalance does not bias the trained policy toward the majority class: it recalls 80.11% of supported and 78.00% of refuted images (Table 1). We also rewrote Sections 5.4 and 5.5 to state their conclusions. If you have any further questions, or if any part of our response is unclear, we would be glad to continue the discussion.

Best regards, The Authors

Reviewer_xfu2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the situation shown in an image is factually supported. The system receives the image and a general verification instruction, without a separate text claim. The authors introduce EviLens, a dataset of 10,174 constructed and web-collected images covering twelve types of factual error. Their EviScene agent inspects images, searches for evidence, and explains its verdict. They train Qwen3.5-9B with supervised fine-tuning followed by privileged on-policy self-distillation. On 1,527 test images, it reaches 79.05% balanced accuracy and 52.26% strict evidence sufficiency, improving over the base agent by 6.71 and 9.37 percentage points.

Critical strengths
  1. EviLens gives the community a useful way to study image factuality beyond detecting generated images. Both the constructed and web-collected subsets contain supported and refuted cases, and answers must be justified with evidence. Two experts check each example for visible grounding and factual support, with a third resolving disagreements. Related images are grouped by source or event when making the splits. The scale of the dataset and the retained evidence records make this a substantial part of the contribution.

  2. The experiments give good evidence that the complete system helps. Training improves the base Qwen3.5-9B agent by 6.71 percentage points in balanced accuracy and 9.37 points in evidence sufficiency. Table 2 shows gains on web-collected images and the two generators excluded from controlled training. Evidence sufficiency also improves across all five backbones when investigation tools are available, and removing tool families hurts performance. These results support the practical value of the approach even without a large improvement from the final training stage.

Critical weaknesses
  1. Evidence sufficiency is scored by a single Gemini 3.7 Flash evaluator, without a reported comparison against human judgments or another evaluator. This leaves some uncertainty about the small differences between the leading systems. The fully trained agent scores 1.77 percentage points higher than both supervised fine-tuning alone and the GPT-5.5 agent, a difference of 27 test cases. I would be cautious about treating this ranking as conclusive without a human check. This concern is mainly about the close comparisons, rather than the larger improvement over the base agent.

  2. The results establish the value of the complete trained agent more clearly than the benefits of each proposed choice. All agents in Table 1 use the EviScene workflow, so the comparison does not isolate the benefit of refining questions over an existing retrieval pipeline adapted to this task. Most of the training gain also comes from supervised fine-tuning. Self-distillation adds 0.24 percentage points in balanced accuracy and 1.77 points in evidence sufficiency while lowering macro F1 by 1.02 points. I therefore see the benchmark and the trained agent as the main contributions. The evidence for self-distillation as a separate contribution is less convincing.

Questions for authors
  1. Has the evidence evaluator been checked against human judgments? Any agreement figures or examples of scoring errors would help interpret the small differences between the leading systems. Does it check citations against the retrieved observations?
  2. What reference date is used for cases such as the museum opening in panel c of Figure 5? How are cases handled when the verdict depends on a date that is not visible in the image?
  3. How many self-distillation rounds, rollouts, and accepted repair and preservation targets were used? Are there estimates of how much the small gain over supervised fine-tuning varies across runs?
3 comments

We thank the reviewer for the careful assessment and address each point below. We have uploaded a revised PDF with changes in blue; all references below are to it.


W1 and Q1: Reliability of the evidence judge

The evidence judge agrees closely with human annotators and with judges from other model families, and Full scores highest under every judge. Following the review protocol used to build EviLens, we sampled 50 responses whose labels match the ground truth from Full, SFT, and the GPT-5.5 agent, half credited by the Gemini judge and half not. Two authors judged each response independently without seeing the judge label, and a third author resolved their disagreements. The two annotators agreed on 44 of the 50 responses, and the final human labels agree with the Gemini judge on 48, or 96%. In one mismatch, the agent correctly refuted an image of a Meta sleeping pod that was never showcased, but also claimed that the infinity logo on the pod does not match Meta's branding, although it is Meta's own logo. Gemini credited the response; the annotators did not, because part of its stated reasoning is false.

We also rescored the saved responses of these three systems on the full test set with Qwen3.8-Max and GLM-5.3 Flash, under the same criteria and reference material.

SESR by judgeGemini 3.7 FlashQwen3.8-MaxGLM-5.3 Flash
GPT-5.5 agent50.4950.6249.77
Qwen3.5-9B, SFT50.4950.6950.43
Qwen3.5-9B, Full52.2652.3951.74
Agreement with Gemini 3.7 Flash—98.69%97.99%

As the reviewer notes, the margin of Full over SFT and the GPT-5.5 agent is small, but it is consistent: Full has the highest SESR under all three judges. Agreement is high because each judgment is anchored to the private reference fact: the judge only checks whether the stated grounds establish that specific fact, which leaves little room for interpretation.

Yes, the judge checks citations against the retrieved observations. Every observation obtained during the investigation, including the evidence extracted from each visited webpage, is saved with the trajectory and passed to the judge together with the image, the private reference, and the final answer. The judge credits a justification only when these observations support it, so invented sources and unsupported claims receive no credit.


W2: Contributions of question refinement and self-distillation

Adaptive question refinement outperforms all three fixed variants and an existing fact-checking pipeline adapted to our task. We compare EviScene with three variants on the full test set in Section 5.3 and Table 4a, all using Gemini 3.7 Flash with the same tools and 24-action budget. The first fixes the verification question before any tool call. The second replaces adaptive investigation with a fixed retrieval pipeline of nine actions: scene perception, OCR, reverse image search, three text searches, and three page visits. The third fixes both. We also adapted DEFAME (Braun et al., 2025), a multimodal fact-checking agent that plans retrieval dynamically for a given claim. Our task provides no claim, so the same backbone first states the claim to verify from the image, and DEFAME then checks it with its own planner and tools.

QuestionAction sequenceBAccSESRAverage actions per image
Adaptive, EviSceneAdaptive81.5742.897.18
FixedAdaptive77.0138.5714.02
AdaptiveFixed78.3239.699.00
FixedFixed75.8336.809.00
Fixed claim, DEFAMEDEFAME planner74.5338.1116.78

EviScene, which adapts both, reaches the highest BAcc and SESR with the fewest actions. Fixing the question lowers BAcc and SESR by 4.56 and 4.32 points while nearly doubling the actions per image, fixing the action sequence lowers them by 3.25 and 3.20 points, and fixing both lowers them by 5.74 and 6.09 points. With its claim fixed at the start, DEFAME trails EviScene by 7.04 BAcc and 4.78 SESR points with more than twice the actions, consistent with the fixed-question variant.

The paper does not present PSD as a separate contribution: our third contribution is the compact policy trained with SFT and PSD. Still, PSD is the only second-stage method we tried that improves over SFT:

Stage after SFTBAccSESR
None, SFT only78.8150.49
Continued SFT on the policy's own correct rollouts77.1449.44
GRPO76.3549.90
PSD, Full79.0552.26

Both alternatives depend on correct rollouts: continued SFT imitates the rollouts the policy already gets right, and GRPO gets no learning signal when every rollout in a group fails. The second-stage training cases are hard, so correct rollouts on them are rare, and neither method learns from the decisions that cause failures; these are exactly what PSD repairs. Q3 reports how the gain of PSD varies across training runs.

Q2: Reference date

All evaluations use a fixed reference date of September 1, 2026, and every verdict is judged against the state of the world on that date. For the museum case in Figure 5c, official records place the building in its construction stage as of that date, so the depicted opening ceremony conflicts with the records. When a verdict depends on a date that is not visible in the image, the depicted situation is likewise assessed against the world as of the reference date. The current date is a runtime parameter: during evaluation we pass the reference date to every system, and when the system is deployed for everyday use, the current date is simply the actual date of each use. We omitted this setting from the paper by oversight and have added it to the evaluation settings in Appendix C.1, Table 14.


Q3: Self-distillation details

PSD ran for one round. Starting from the SFT policy, we ran 6 investigations on each of 1,400 training cases, 8,400 in total. Each training target is one decision of the policy together with its context: a repair target is a corrected decision that passed review, and a preservation target is a decision taken unchanged from a successful investigation. Every decision in a successful investigation is a preservation candidate, whereas a failed investigation contributes only its repaired decisions, so preservation candidates far outnumber repairs. We balanced the two types and trained the policy on 1,598 repair and 1,598 preservation targets, 3,196 in total, for 5 epochs.

The gain of PSD holds in every training run. We trained PSD three times from the same SFT policy on the same targets with different random seeds and evaluated each run on the full test set.

PolicyBAccSESR
SFT78.8150.49
Full, three PSD runs79.81 ± 0.8652.02 ± 0.31

Full exceeds SFT in SESR in every run, by 1.18 to 1.77 points, and its mean BAcc of 79.81 is above both SFT and the run reported in the paper.

Dear Reviewer xfu2,

Thank you again for raising the reliability of the evidence judge and the contribution of self-distillation. In our response, a human check of 50 sampled responses agreed with the Gemini judge on 48, two judges from other model families agreed with it on 98.69% and 97.99% of cases, and Full has the highest SESR under all three. PSD is the only second-stage method we tried that improves over SFT, and all three PSD runs exceed SFT in SESR. If you have any further questions, or if any part of our response is unclear, we would be glad to continue the discussion.

Best regards, The Authors

Reviewer_sSnLOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies image factuality assessment beyond identifying whether an image is AI-generated. Given an image and a generic verification instruction, without an accompanying textual claim, the system must identify the facts to investigate, align external evidence with the depicted situation, and produce an evidence-grounded verdict. The authors introduce EviLens, a benchmark of 10,174 images covering twelve factual error types, with evaluation of both verdict correctness and evidence sufficiency. The proposed EviScene framework interleaves visual inspection, external retrieval, and evidence analysis. Retrieved information guides question refinement and visual reinspection, while new visual findings inform subsequent investigation. Training combines full-trajectory supervised fine-tuning with privileged on-policy self-distillation (PSD), using verified local repairs from the policy’s own investigations alongside preservation targets from successful trajectories. Experiments include multiple backbones, training-stage comparisons, tool ablations, and analyses across image sources and generators. The full Qwen3.5-9B policy achieves 79.05% balanced accuracy and 52.26% Strict Evidence Sufficiency Rate (SESR), with the highest SESR among the evaluated agents.

Critical strengths
  1. The task formulation is valuable, and the paper provides a systematic dataset resource. The paper distinguishes how an image was produced from whether its depicted content is factually supported. Requiring models to identify what needs verification and ground their judgments in external evidence connects visual understanding with factual investigation in a meaningful way. EviLens covers diverse factual errors, combines constructed and web-collected images, and includes expert review, factual explanations, and provenance records. These materials support evaluation of both predictions and their evidential basis. I consider the task formulation and dataset important contributions that provide a useful foundation for subsequent work.
  2. The method and training design are well aligned with the decisions required for multi-step investigation. EviScene allows new evidence to refine verification questions and motivate reinspection of relevant image details. Full-trajectory SFT establishes investigation capabilities, while PSD targets recoverable errors in the policy’s own investigations using verified hint-assisted continuations. Acceptance of a repair requires correction of the selected decision, a correct final verdict, and support from the evidence actually obtained. This extends supervision beyond final labels to search direction, factual association, and evidence interpretation. Multi-backbone comparisons, training-stage analyses, and tool ablations provide empirical support, while source- and generator-specific results broaden the evaluation. I appreciate the task-driven methodological innovation in combining adaptive question refinement, visual evidence reinspection, and local decision repair for autonomous image factuality assessment.
Critical weaknesses
  1. The independent contribution of adaptive investigation is not fully established. Refining verification questions and revisiting image details based on intermediate evidence are central to EviScene. The tool ablations demonstrate the usefulness of retrieval and evidence inspection, but do not directly isolate the additional benefit of adapting the investigation. A simpler workflow that determines its verification questions in advance could also benefit from the same tools. A comparison against such a workflow would help quantify the additional benefit of adaptive investigation and identify the cases in which it is most useful.
  2. The reliability of binary judgments under insufficient evidence is not directly established. When retrieval yields no useful evidence, sources conflict, or the investigation budget is exhausted, the system may still lack evidence for the decisive fact but must output Supported or Refuted. Although the authors explicitly instruct the model to distinguish insufficient evidence from factual falsity, the experiments do not establish how often these situations occur or how reliable the resulting judgments are. This limits our understanding of the method’s performance on difficult-to-verify cases.
Questions for authors
  1. What is the additional benefit of dynamically refining verification questions? The existing tool ablations establish the value of retrieval and evidence inspection, but do not isolate the benefit of adapting the investigation based on intermediate evidence. Could the authors compare EviScene with a workflow that fixes its verification questions before evidence collection, using the same backbone and tool access under a comparable action budget? This would help quantify the contribution of adaptive investigation.
  2. How reliable are the final judgments when decisive evidence remains unavailable? Could the authors analyze a representative subset of existing investigation trajectories to report how often investigations end without decisive evidence, together with verdict accuracy and the main error types in these cases? Please explain how evidence insufficiency is identified, distinguishing failure to obtain evidence from failure to use or report evidence already retrieved. Representative examples would help illustrate the resulting judgments.
  3. Are the SESR results robust to the choice of judge model? SESR is currently evaluated using Gemini 3.7 Flash. Could the authors rescore the same saved responses with a judge from another model family, using the same evaluation criteria and reference material? Please report inter-judge agreement and whether the main model rankings and the Full-versus-SFT SESR improvement remain consistent. A representative test subset would also be informative.
  4. How sensitive is the method to the investigation budget? Could the authors compare action limits such as 8, 16, and 24 on the same test subset, reporting BAcc, SESR, the average number of actions used, and the proportion of investigations that reach the limit? This would clarify whether additional actions improve verdict accuracy and evidence sufficiency, and help characterize the trade-off between performance and investigation cost.
5 comments

We thank the reviewer for the constructive questions and address each point below. We have uploaded a revised PDF with changes in blue; all references below are to it.


W1 and Q1: Benefit of adaptive investigation

Adapting the investigation to intermediate evidence improves BAcc and SESR over workflows that fix the verification question, the action sequence, or both. Section 5.3 and Table 4a compare these on the full test set, all using Gemini 3.7 Flash with the same tools and 24-action budget. With a fixed question, the agent states one verification question from the image before any tool call and keeps it throughout. With a fixed sequence, every image goes through the same nine actions: scene perception, OCR, reverse image search, three text searches, and three page visits; the agent only fills in the tool arguments.

QuestionAction sequenceBAccSESRAverage actions per image
Adaptive, EviSceneAdaptive81.5742.897.18
FixedAdaptive77.0138.5714.02
AdaptiveFixed78.3239.699.00
FixedFixed75.8336.809.00

EviScene, which adapts both, reaches the highest BAcc and SESR with the fewest actions. Fixing the question lowers BAcc and SESR by 4.56 and 4.32 points while nearly doubling the actions per image, fixing the action sequence lowers them by 3.25 and 3.20 points, and fixing both lowers them by 5.74 and 6.09 points. Adaptivity is most useful when the image does not reveal at the outset which detail is decisive: a question fixed before any evidence commits the investigation to one detail, whereas EviScene revises what it checks as findings accumulate. Appendix D.2 describes the variants.


W2: Reliability of binary judgments under insufficient evidence

The three situations differ in how often they occur. Running out of actions is rare with the 24-action budget: only 0.8% of investigations reach the limit (Q4). Conflicting sources are the cause in only 0.4% of the investigations that end without decisive evidence. The main case is retrieval that yields no decisive evidence, which covers 29.9% of investigations. Judgments in these cases are much less reliable than those backed by evidence, with 43.0% versus 93.9% of labels matching the ground truth. SESR credits none of them and therefore reports separately the verdicts that rest on decisive evidence.


Q2: Analysis of investigations without decisive evidence

We analyzed all 1,527 test investigations of Qwen3.5-9B Full under the 24-action budget.

Identification. For each investigation, Gemini 3.7 Flash reads the private reference fact, every tool observation, and the final report, and decides whether the observations settle the decisive fact and whether the report relies on them. This separates failure to obtain evidence from failure to use or report it.

Frequency and accuracy. The table reports how often each case occurs and how often its label matches the ground truth; every case credited by SESR falls in the first row.

Decisive evidenceShare of investigationsLabel matches ground truth
Obtained and used63.1%93.9%
Obtained but not used or misread7.1%91.7%
Not obtained29.9%43.0%

Main error types. Among investigations without decisive evidence, 58.8% search for the decisive detail without finding relevant results, 40.4% never target it, and access failures and conflicting sources account for 0.4% each.

Examples.

TypeImageDifficultyOutcome
Not obtainedUNESCO experts document damage to the Derzhprom building in Kharkiv; the mission took place on February 28, 2025Searches return UNESCO's general 3D scanning work in Ukraine, but neither the building nor the Kharkiv missionThe agent accepted the image from this related context: the label matches without decisive evidence
Obtained but misreadA track team with a second-place trophy; it won both team titlesA day-two report ranks the team second, while a later post congratulates it as championThe agent trusted the day-two report: the label does not match

Q3: Robustness to the judge model

SESR is robust to the judge model: two judges from other model families agree with Gemini 3.7 Flash on 98.69% and 97.99% of per-case decisions, and Full scores highest under all three. We rescored the saved responses of the three leading agents on the full test set with Qwen3.8-Max and GLM-5.3 Flash, using the same criteria and reference material.

SESR by judgeGemini 3.7 FlashQwen3.8-MaxGLM-5.3 Flash
GPT-5.5 agent50.4950.6249.77
Qwen3.5-9B, SFT50.4950.6950.43
Qwen3.5-9B, Full52.2652.3951.74
Agreement with Gemini 3.7 Flash—98.69%97.99%

Full's gain over SFT is 1.77, 1.70, and 1.31 points under the three judges. Agreement is high because each judgment is anchored to the private reference fact: the judge only checks whether the stated grounds establish that specific fact, which leaves little room for interpretation.


Q4: Sensitivity to the action budget

Establishing the decisive fact takes several rounds of investigation. We reran the trained Qwen3.5-9B policy on the full test set with limits of 8 and 16 actions; the limit of 24 is the setting reported in the paper. We add this comparison to Section 5.3 and Table 4b.

Action limitBAccSESRAverage actions per imageInvestigations reaching the limit
876.7636.547.3363.6%
1677.5548.5911.066.6%
2479.0552.2612.050.8%

Raising the limit from 8 to 16 adds 12.05 SESR points and 0.79 BAcc points; raising it further to 24 adds 3.67 SESR and 1.50 BAcc points for about one more action per image on average. With 24 actions, the policy ends 99.2% of investigations on its own, so the reported limit leaves enough room for investigation.

When an investigation reaches the limit, its retained history, including every observation obtained so far, passes to the same terminal judgment that follows a completed investigation (Appendix B.2). At the limit of 24, only 12 investigations reach it; 8 of their labels match the ground truth, and none meets SESR. At the limit of 8, a Gemini 3.7 Flash analysis shows that 12.67% of truncated investigations had already retrieved the decisive evidence and received a label matching the ground truth.

Dear Reviewer sSnL,

Thank you again for your questions on adaptive investigation and on judgments without decisive evidence. In our response, fixing the verification question before investigation lowered BAcc and SESR by 4.56 and 4.32 points while nearly doubling the actions per image (Table 4a). Across all 1,527 test investigations, 93.9% of labels backed by decisive evidence match the ground truth, compared with 43.0% when that evidence is not obtained, and SESR credits only the former. If you have any further questions, or if any part of our response is unclear, we would be glad to continue the discussion.

Best regards, The Authors

Comments on the paper

2 comments

We thank the reviewers and the AC for their time. The reviewers find the task formulation valuable and EviLens a substantial resource (sSnL, xfu2), recognize the large manually reviewed dataset and extensive evaluation (2KqQ), and note that the complete system clearly helps across backbones (xfu2). Below we summarize the main concerns and our responses.


1. Does adaptive investigation help? (2KqQ, sSnL, xfu2)

Yes. We compared EviScene with variants that fix the verification question, the action sequence, or both, using the same Gemini 3.7 Flash backbone, tools, and 24-action budget on the full test set.

QuestionAction sequenceBAccSESRActions per image
Adaptive, EviSceneAdaptive81.5742.897.18
FixedAdaptive77.0138.5714.02
AdaptiveFixed78.3239.699.00
FixedFixed75.8336.809.00

EviScene reaches the highest BAcc and SESR with the fewest actions. A fixed question nearly doubles the actions per image while lowering both metrics. An adapted DEFAME pipeline, which first states a claim from the image and then verifies it, also falls behind, with 74.53% BAcc and 38.11% SESR at 16.78 actions per image.


2. Is the evidence judge reliable? (sSnL, xfu2)

Yes. In a blind check of 50 responses, half credited by the judge and half not, human labels agree with the Gemini judge on 48. Two judges from other model families, Qwen3.8-Max and GLM-5.3 Flash, agree with it on 98.69% and 97.99% of per-case decisions, and the full policy has the highest SESR under all three judges.


3. How reliable are verdicts without decisive evidence, and why is direct judgment's SESR low? (2KqQ, sSnL)

Verdicts are reliable when they rest on decisive evidence: the full Qwen3.5-9B policy obtains and uses such evidence in 63.1% of its investigations, and 93.9% of these labels match the ground truth. The remaining investigations mostly search for the decisive detail without relevant results or never target it, and SESR credits none of them, so it separates verdicts grounded in evidence from the rest. Direct judgment is the extreme case: without investigation, a matching label rarely comes with grounds for the decisive fact, so its SESR stays low even when its BAcc approaches that of agents.


4. How sensitive is the method to the action budget? (sSnL)

Establishing the decisive fact takes several rounds of investigation, and the 24-action limit leaves enough room. SESR rises from 36.54% at 8 actions to 48.59% at 16 and 52.26% at 24, and at 24 actions the policy ends 99.2% of investigations on its own.


5. What does self-distillation contribute? (xfu2)

PSD is part of how we train the compact policy rather than a separate contribution, but it is the only second-stage method we tried that improves over SFT: continued SFT on the policy's own correct rollouts and GRPO reach 49.44% and 49.90% SESR, compared with 50.49% for SFT and 52.26% for PSD. Across three PSD runs with different seeds, the policy reaches 79.81 ± 0.86 BAcc and 52.02 ± 0.31 SESR, and every run exceeds SFT in SESR.


6. Task definition, label balance, and presentation (2KqQ, xfu2)

  1. Task. Multimodal fact-checking verifies a given claim, and implicit claims are first extracted in a separate stage. Image Factuality Assessment supplies no claim and has no extraction stage; the system decides during investigation which depicted facts to check.
  2. Label balance. The imbalance does not affect the results: BAcc weights both classes equally, the full policy recalls 80.11% of supported and 78.00% of refuted images, and on the balanced web-collected test images it reaches 77.21% BAcc, compared with 76.75% on constructed images.
  3. Analysis sections. Sections 5.4 and 5.5 now lead with their conclusions: training gains extend to web-collected images and to generators unseen in training, and prior knowledge and visual plausibility cannot substitute for evidence about the depicted situation.
  4. Reference date. All evaluations use a fixed reference date of September 1, 2026, which the paper now states in Appendix C.1.

We thank the reviewers again for comments that strengthened the paper.

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their thoughtful and constructive feedback, which helped us further clarify our results. As the discussion period closes, we briefly summarize the discussion.

Strengths recognized by the reviewers.

  • Task. The task formulation is valuable: it distinguishes whether the depicted content is factually supported from how the image was produced, and connects visual understanding with factual investigation (sSnL, xfu2).
  • Dataset. EviLens is a substantial resource: it is large and manually reviewed, combines constructed and web-collected images covering diverse factual errors, and retains expert-reviewed explanations, evidence, and provenance records (2KqQ, sSnL, xfu2).
  • Method. The method and training design are well aligned with the decisions required for multi-step investigation, combining adaptive question refinement, visual reinspection, and local decision repair (sSnL).
  • Experiments. The evaluation is extensive across open- and closed-source models, and gives good evidence that the complete system helps: training improves the base agent by 6.71 BAcc and 9.37 SESR points, and evidence sufficiency improves across all five backbones (2KqQ, sSnL, xfu2).

Reviewer sSnL confirmed after the discussion that our responses addressed all of their concerns about adaptive investigation, judgments under insufficient evidence, judge robustness, and the action budget, and maintained their rating of 4, the highest on the scale.

Reviewer 2KqQ. Our rebuttal addresses every weakness and question raised by the reviewer:

  • Agent architectures (W2). We compared EviScene with variants that fix the verification question, the action sequence, or both, under the same backbone, tools, and action budget. EviScene achieves the best BAcc and SESR with the fewest actions. Fixing them lowers BAcc by 3.25 to 5.74 points and SESR by 3.20 to 6.09 points, and fixing the question also nearly doubles the actions per image (Section 5.3, Table 4a).
  • Label imbalance (W1). The imbalance does not bias the results. BAcc weights both classes equally, and the trained policy recalls 80.11% of supported and 78.00% of refuted images (Table 1).
  • Analysis sections (W3, Q2). Sections 5.4 and 5.5 now lead with their conclusions: training gains extend to web-collected images and to generators unseen in training, and prior knowledge and visual plausibility cannot substitute for evidence about the depicted situation.
  • Task distinction (Q1). Multimodal fact-checking verifies a given claim, and implicit claims are first extracted in a separate stage. Image Factuality Assessment supplies no claim and has no extraction stage; the system decides during investigation which depicted facts to check.
  • SESR gap (Q3). The gap is expected. BAcc checks only the label, whereas SESR credits a response only when its stated grounds establish the decisive fact behind the verdict. Under direct judgment, the explanation often rests on cues such as a rendering artifact or an unrelated fact rather than the decisive fact, so SESR stays low even when the label matches the ground truth. Section 5.2 now explains this.

Reviewer xfu2. Our rebuttal addresses every weakness and question raised by the reviewer:

  • Evidence judge (W1, Q1). A human check of 50 sampled responses agrees with the Gemini judge on 48. Two judges from other model families agree with it on 98.69% and 97.99% of cases, and Full has the highest SESR under all three judges. The judge also checks the stated grounds against the observations retrieved during the investigation.
  • Question refinement (W2). Beyond the fixed variants above, EviScene also outperforms an adapted DEFAME pipeline, an existing multimodal fact-checking agent, with 81.57% BAcc and 42.89% SESR against 74.53% and 38.11%, while using fewer than half the actions.
  • Self-distillation (W2, Q3). PSD is part of how we train the compact policy rather than a separate contribution, but it is the only second-stage method we tried that improves over SFT. Across three PSD runs with different seeds, the policy reaches 52.02 ± 0.31 SESR against 50.49 for SFT, and every run exceeds SFT. We also reported the number of rounds, rollouts, and training targets.
  • Reference date (Q2). All evaluations use a fixed reference date of September 1, 2026, now stated in Appendix C.1, Table 14.

We believe these results establish Image Factuality Assessment as a well-defined task, and EviLens and EviScene as a solid foundation for evidence-grounded visual verification. We thank the reviewers and the AC again for their time and engagement.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.